Nn models sets how to#
Nn models sets update#

When it comes to finally swapping faces, we switch the decoders around, therefore we feed the model Face A, but pass it through Decoder B. Decoder B is taking the encoding vectors and attempting to recreate Face B. Decoder A is taking the encoding vectors and attempting to recreate Face A. Switched Decoders - When training the model, we train 2 decoders.The encoder therefore needs to see, and learn, both sets of faces that we require for the swap. This is extremely important, as we will ultimately be telling our Neural Network to take the encodings of one face and decode it to another face. This way our encoder is learning a single algorithm for 2 different people. The first step to achieve this is sharing the Encoder for both the A and B set. The A set (the original faces that we want to replace) and the B set (the swap faces that we wish to place in a scene). Shared Encoder - When we train our model, we are feeding it 2 sets of faces.To achieve this, our NN does a couple of things: we want to take a load of faces and reconstruct someone else's face.

Nn models sets how to#

Now we have the basics of what a Neural Network does and how it learns to create faces, how does this apply to face swapping? You may have noticed in the above breakdown that this NN learns how to take a load of faces of a person and then reconstruct those faces. The model then repeats this action many, many times constantly updating its weights based on its loss values, theoretically improving over time, until it reaches a point where you feel it has learned enough to effectively recreate a face, or the loss values stop falling. If it feels it has improved, then it knows to keep adjusting the weights in the direction that it is going. If it has adjusted its weights in one direction, but feels it has done a worse job of reconstructing the face than previously, then it knows that the weights are moving in the wrong direction, so it will adjust them the other way. These feed into the Encoder/Decoder algorithms. Weights - Once the model has evaluated how well it has recreated a face it updates its weights.

Nn models sets update#

Based on how well it thinks it has done, it will give itself a score (the loss value) and will update its weights accordingly. Loss - For every batch of faces fed into the model, the NN will look at the face it has attempted to recreate by its current encoding and decoding algorithm and compare it to the actual face that was fed in.The NN needs to know how well it is doing encoding and decoding faces. Some models are constructed slightly differently, but the basic premise remains the same. Decoder - This has the job of taking the vectors created by the encoder and attempting to turn this representation back into faces, as closely matching the input images as possible.It is important to note that it is not learning an exact representation of every face you feed into it, rather, it is trying to create an algorithm that can be used to later reconstruct faces as closely as possible to the input images. Encoder - This has the job of taking a load of faces as an input and 'encoding' them into a representation in the form of a 'vector'.Most of the models are largely made up of 2 parts: I will be using the GUI for this guide, but the premise is exactly the same for the cli (all the options present in the GUI are available in the cli).Īt a high level, training is teaching our Neural Network (NN) how to recreate a face. Having a basic understanding about how the Neural Network works, and the kind of data it benefits from seeing will vastly improve your chances of achieving a successful swap. Machine Learning is a complicated concept, but I have tried to break it down to be as simple to understand as possible. I advise that you familiarize yourself with it all. There is quite a lot of background information in this guide.

If you have not already generated your face sets for training, then stop right now and head over to the Extract Guide to generate them now. I won't cover everything off, but hopefully this will give you enough to make informed decisions of your own. There are several models with many options. In this post I will detail how we train a model. It's how we learn, but it can sometimes help to have a bit of an understanding of the processes involved before diving in. I will try to keep it updated.Ī lot of people get overwhelmed when they start out Faceswapping, and many mistakes are made. NB: This guide was correct at time of writing, but things do change.