How a face swap actually works

Stage 1 — Detection

The first job is finding faces in the target image. A detector scans the frame and returns a bounding box for each one, along with a confidence score.

This is why group photos work: every face is found, and you pick which one to replace rather than the tool choosing for you. It is also the first place things fail. A face that is very small in frame, heavily shadowed, or turned close to profile may not be detected at all — and a face that is not detected cannot be swapped.

Stage 2 — Landmark alignment

Once a face is located, the model maps facial landmarks: eye corners, pupil centres, nose bridge and tip, mouth corners, jaw outline. Typically dozens of points, sometimes hundreds.

These points are found on both the source face and the target face, and the source is then warped so its geometry matches the target's pose. If the target's head is tilted fifteen degrees and turned slightly away, the source face is transformed to sit at that same angle.

This stage is why a front-facing source photo matters so much. Warping a front-facing face into a moderate angle is a small transformation. Warping a profile shot into a front-facing pose means inventing the half of the face that was never photographed.

Stage 3 — Identity transfer

Here the generative model does the actual work: producing a new face that carries the source person's identity — the features that make them recognisably them — while keeping the target's expression, gaze, and head position.

That split is the whole trick. If it kept too much of the source, you would get a face wearing the wrong expression, staring in the wrong direction. If it kept too much of the target, you would get the original person with slightly different features. The balance is what separates a convincing swap from an uncanny one.

It is also why cross-gender and cross-age swaps work: the model is transferring identity, not matching demographic categories.

Stage 4 — Compositing

The generated face now has to become part of the photograph. This stage handles:

• Colour and tone matching — adjusting the face to the scene's white balance and exposure, so it does not read as lighter, warmer, or flatter than the neck beneath it.
• Lighting direction — placing shadow consistent with where the light in the scene is coming from.
• Edge blending — feathering the boundary along the hairline, temples, and jaw, which is the hardest edge in the whole process.
• Grain and sharpness matching — a clean, smooth face on a grainy photo looks wrong even when everything else is right.

Most tools that produce obviously fake results do stages one to three competently and skip most of stage four. If you have ever looked at a swap and known instantly that something was off without being able to say what, the answer is almost always here.

What changes for video

Video runs all four stages on every frame, and then adds a fifth consideration: consistency between frames. Because the model makes marginally different choices each time it runs, independently swapped frames produce a face that shimmers and crawls during playback. Temporal smoothing compares neighbouring frames and stabilises identity, edges, and colour across them.

That extra pass is why video takes substantially longer than the same number of standalone photo swaps. More on video swapping.

What this means for your photos

Practical checklists are on the photo swap and video swap pages.