A faster face swap engine, up to 4.4× quicker per frame
We rebuilt the pipeline that renders every frame of your stream. It does less work, keeps each frame on the GPU, and fixes washed-out skin tones.
Every frame of a Livesync stream goes through the same steps: find the face, map it, swap it, clean it up, and send it on. We rebuilt that pipeline from the ground up. It now does less work per frame, keeps the frame on the GPU from start to finish, and renders skin tones the way they should look.
Up to 4.4× less time per frame
The time it takes to render one frame decides how smooth and how close to real time your stream feels. On the same GPU we run in production, the new engine needs a fraction of the time the old one did at every pixel boost setting.
What changed
- Fewer models per frame. We went from seven models running on every frame to four, removing analysis whose results were never used.
- The whole frame stays on the GPU. Each frame is now uploaded once and downloaded once. Before, it bounced between the CPU and GPU several times and was blended back on the CPU.
- Batched, mixed-precision face swap. All tiles of a face are swapped in a single GPU call, and the swap model runs in mixed precision. That step alone went from 21.6 ms to 6.6 ms with no visible difference.
- Leaner worker. We removed per-frame process coordination, a cache that grew without limit, and a messaging leak, and gave each worker a second CPU core.
Smoother live streams
In a real streaming room, the final tuning took the viewer's frame rate from 22 to 24 fps up to 29 fps from a 30 fps camera, and the share of frames that reach the viewer from about 83% to 96%. The longest pauses between frames got shorter too.
Truer skin tones
We also found and fixed a colour channel mix-up that made swapped faces look pale and slightly grey. Faces now keep natural, warmer skin tones, and the result stays a little closer to the source persona than before.
One limit we set
At pixel boost 512 and above, no single GPU we tested kept up in real time, so 2K and 4K plans now stream at pixel boost 256. You get a steady stream instead of one that stutters.
How we measured
Per-frame timings come from 135 different 1280×720 frames on an RTX 4090, the GPU we run in production, after a short warm-up and with the process limited to one CPU core. Our timing of the previous engine leaves out some of its overhead, so the real gap is slightly larger than shown. Live figures come from a 60 second run in a real streaming room with a 720p, 30 fps camera. Most of the delay you see on a live stream is the network between you, the GPU and your viewers, so it depends on where everyone is. When demand is high, a stream can occasionally run on a different GPU model, where frame rates are lower than the figures above.