LeapTalk: Breaking the Latency-Quality Trade-off in Talking Head Generation

• URL: http://arxiv.org/abs/2608.00079v1
• Date: Wed, 29 Jul 2026 14:44:57 GMT
• Title: LeapTalk: Breaking the Latency-Quality Trade-off in Talking Head Generation
• Abstract summary: We propose LeapTalk, a novel framework that achieves stable and real-time talking-head generation with a single forward step.<n>Our method achieves high-fidelity and temporally consistent video generation with only 1 step at up to 200 FPS.
• Score: 28.818825505825476
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Long-form and real-time talking-head generation remains challenging due to a latency-quality trade-off: inefficient multi-step diffusion prohibits streaming generation, whereas real-time autoregressive approaches suffer from error accumulation and identity drift. To address this drawback, we propose LeapTalk, a novel framework that achieves stable and real-time talking-head generation with a single forward step, scaling to arbitrarily long videos. At the heart of our approach lies a single-step bridge distillation scheme. On the one hand, departing from the conventional noise-to-data paradigm, we introduce a data-to-data transport formulation based on a Brownian bridge. Anchored by a persistent reference, this strategy effectively mitigates identity drift and enhances long-term temporal stability. On the other hand, to enable smooth knowledge transfer from a pre-trained diffusion teacher to the student bridge model, we explore a heterogeneous distillation framework with an SNR-aligned time transformation $Φ(τ)$, which bridges the functional discrepancy between the two models. Moreover, we propose an audio-driven classifier-free guidance mechanism to maintain fine-grained lip synchronization under extreme step reduction. Extensive experiments demonstrate that our method achieves high-fidelity and temporally consistent video generation with only 1 step at up to 200 FPS, significantly outperforming existing approaches in both efficiency and stability. Project Page: https://zhangrongxiang.github.io/leaptalk-page/

Related papers

• FlowLong: Inference-time Long Video Generation via Manifold-constrained Tweedie Matching [68.01498128172214]
  We propose a novel but simple inference-time approach for long video generation that is architecture-agnostic and requires no additional training.<n>Our method generates long videos via overlapping sliding windows, where predicted clean samples from adjacent windows are blended via emphTweedie matching to enforce both textbfmanifold constraint and temporal consistency across overlap regions.<n>Our method generates videos several times longer than the native window length while outperforming both training-free and autoregressive baselines in temporal consistency and visual quality, and further extends to audio-video joint generation and text-to-3DGS without any
  arXiv  Detail & Related papers  (2026-05-20T08:55:37Z)
• OmniForcing: Unleashing Real-time Joint Audio-Visual Generation [51.031726911410594]
  Recent joint audio-visual diffusion models achieve remarkable generation quality but suffer from high latency.<n>We propose OmniForcing, the first framework to distill an offline, dual-stream bidirectional diffusion model into a high-fidelity streaming autore generator.
  arXiv  Detail & Related papers  (2026-03-12T08:17:36Z)
• Streaming Autoregressive Video Generation via Diagonal Distillation [50.13573884115673]
  Autoregressive models offer a natural framework for sequential frame synthesis but require heavy computation to achieve high fidelity.<n>We propose Diagonal Distillation to exploit temporal information across both video chunks and denoising steps.<n>Our method generates a 5-second video in 2.61 seconds (up to 31 FPS), achieving a 277.3x speedup over the undistilled model.
  arXiv  Detail & Related papers  (2026-03-10T10:45:24Z)
• HiAR: Efficient Autoregressive Long Video Generation via Hierarchical Denoising [52.237486207964245]
  HiAR is a hierarchical denoising framework that reverses the conventional generation order.<n>It performs causal generation across all blocks at every denoising step, so that each block is always conditioned on context at the same noise level.<n>On VBench (20s generation), HiAR achieves the best overall score and the lowest temporal drift among all compared methods.
  arXiv  Detail & Related papers  (2026-03-09T17:58:16Z)
• TempoSyncDiff: Distilled Temporally-Consistent Diffusion for Low-Latency Audio-Driven Talking Head Generation [0.0]
  This paper introduces TempoSyncDiff, a reference-conditioned latent diffusion framework.<n>It explores few-step inference for efficient audio-driven talking-head generation.<n>The framework incorporates identity anchoring and temporal regularization designed to mitigate identity drift and frame-to-frame flicker.
  arXiv  Detail & Related papers  (2026-03-06T09:09:01Z)
• Predict to Skip: Linear Multistep Feature Forecasting for Efficient Diffusion Transformers [10.751183015853863]
  Diffusion Transformers (DiT) have emerged as a widely adopted backbone for high-fidelity image and video generation.<n>We propose textbfPrediT, a training-free acceleration framework that formulates feature prediction as a linear multistep problem.<n>Our method achieves up to $5.54times$ latency reduction across various DiT-based image and video generation models, while incurring negligible quality degradation.
  arXiv  Detail & Related papers  (2026-02-20T09:33:59Z)
• EchoTorrent: Towards Swift, Sustained, and Streaming Multi-Modal Video Generation [8.795438456031512]
  Multi-modal generation models have achieved high visual quality, but their prohibitive latency and limited temporal stability hinder real-time deployment.<n> Streaming inference exacerbates these issues, leading to pronounced multimodal ambiguities, such as blurring, temporal drift, and lip dechronization.<n>We propose EchoTorrent, a novel novel with a fourfold schema: Multi-Teacher Training fine-tunes a pre-trained model on distinct preference domains; Adaptive DMD (ACCDMD) calibrates the audio CFG degradation errors in phased via a schedule; Long Hybrid Tail, which enforces alignment exclusively on tail frames during long-horizon self-roll
  arXiv  Detail & Related papers  (2026-02-14T08:32:38Z)
• LoL: Longer than Longer, Scaling Video Generation to Hour [50.945885467651216]
  This work achieves the first demonstration of real-time, streaming, and infinite-length video generation with little quality decay.<n>As an illustration, we generate continuous videos up to 12 hours in length, which, to our knowledge, is among the longest publicly demonstrated results in streaming video generation.
  arXiv  Detail & Related papers  (2026-01-23T17:21:35Z)
• SoulX-FlashTalk: Real-Time Infinite Streaming of Audio-Driven Avatars via Self-Correcting Bidirectional Distillation [16.34443339642213]
  textbfX-FlashTalk is a 14B-scale system to achieve a textbfsub-second start-up latency (0.87s) while reaching a real-time throughput of textbf32 FPS.<n>SoulX-FlashTalk is the first 14B-scale system to achieve a textbfsub-second start-up latency (0.87s) while reaching a real-time throughput of textbf32 FPS.
  arXiv  Detail & Related papers  (2025-12-29T11:18:24Z)
• Towards One-step Causal Video Generation via Adversarial Self-Distillation [71.30373662465648]
  Recent hybrid video generation models combine autoregressive temporal dynamics with diffusion-based spatial denoising.<n>Our framework produces a single distilled model that flexibly supports multiple inference-step settings.
  arXiv  Detail & Related papers  (2025-11-03T10:12:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.