AsymTalker: Identity-Consistent Long-Term Talking Head Generation via Asymmetric Distillation

• URL: http://arxiv.org/abs/2605.02948v2
• Date: Fri, 08 May 2026 17:11:57 GMT
• Title: AsymTalker: Identity-Consistent Long-Term Talking Head Generation via Asymmetric Distillation
• Abstract summary: AsymTalker is a novel diffusion-based talking head generation method.<n>It mitigates temporal-spatial misalignment by transforming the static identity image into a temporally coherent latent representation.<n>It guarantees high-fidelity, identity-consistent synthesis over 600-second videos and reaches a real-time inference speed of 66 FPS.
• Score: 13.60100129586038
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Diffusion-based talking head generation has achieved remarkable visual quality, yet scaling it to long-term videos remains challenging. The widely adopted chunk-wise paradigm introduces two fundamental failures: (1) temporal-spatial misalignment between static identity references and dynamic audio streams, and (2) cascading identity drift propagated through self-generated continuity references across chunks. To address both issues, we propose AsymTalker, a novel diffusion-based talking head generation method comprising Temporal Reference Encoding (TRE) and Asymmetric Knowledge Distillation (AKD). First, TRE mitigates temporal-spatial misalignment by transforming the static identity image into a temporally coherent latent representation through encoding of a temporally replicated pseudo-video, without introducing additional parameters. Second, AKD resolves the inherent conditioning dilemma in chunk-wise training: using ground-truth references causes train-inference mismatch, while self-generated references entangle supervision with identity drift. Our asymmetric design circumvents this by anchoring the teacher model with ground-truth continuity references to provide drift-free, chunk-level supervision, thereby avoiding the teacher bottleneck. Meanwhile, the student model learns under inference-aligned conditions, conditioned only on self-generated references, and is trained via distribution matching to preserve identity over long horizons. Extensive experiments show AsymTalker achieves state-of-the-art results on HDTF and VFHQ. It guarantees high-fidelity, identity-consistent synthesis over 600-second videos and reaches a real-time inference speed of 66 FPS.

Related papers

• LeapTalk: Breaking the Latency-Quality Trade-off in Talking Head Generation [28.818825505825476]
  We propose LeapTalk, a novel framework that achieves stable and real-time talking-head generation with a single forward step.<n>Our method achieves high-fidelity and temporally consistent video generation with only 1 step at up to 200 FPS.
  arXiv  Detail & Related papers  (2026-07-29T14:44:57Z)
• Self Gradient Forcing: Native Long Video Extrapolation [60.00513917489133]
  Self Gradient Forcing restores historical context-gradient gap.<n>Code and models will be released to advance research on autoregressive video generation.
  arXiv  Detail & Related papers  (2026-07-22T16:59:23Z)
• Distilling Temporal Coherence into 2D Networks for Transrectal Ultrasound Prostate Video Segmentation [2.65131397683555]
  Real-time video segmentation of the prostate in Transrectal Ultrasound (TRUS) is essential for image-guided interventions.<n>We present a Temporally Consistent Learning Framework that distills temporal coherence into a 2D network during training.<n>Our design is driven by a key clinical observation: the prostate exhibits geometric stability, whereas the surrounding acoustic environment fluctuates due to physiological motion and transducer pressure.
  arXiv  Detail & Related papers  (2026-06-30T06:27:02Z)
• Spatial-Temporal Decoupled Reference Conditioning for Identity-Preserving Text-to-Video Generation [79.94088803584262]
  Identity-preserving video generation (IPVG) aims to synthesize high-fidelity videos that follow text prompts while preserving a reference identity.<n>We propose ST-DRC, an effective Spatial-Temporal Decoupled Reference Conditioning framework for identity-preserving text-to-video generation.<n>We show that ST-DRC achieves strong identity preservation, prompt alignment, temporal consistency, and video quality with a lightweight design built on LTX-2.3.
  arXiv  Detail & Related papers  (2026-06-01T16:12:18Z)
• CoSyncDiT: Cognitive Synchronous Diffusion Transformer for Movie Dubbing [76.74048814837336]
  Movie dubbing aims to synthesize speech that preserves the vocal identity of a reference audio while synchronizing with the lip movements in a target video.<n>Existing methods fail to achieve precise lip-sync and lack naturalness due to explicit alignment at the duration level.<n>We propose a novel flow matching-based movie dubbing framework driven by the Cognitive Synchronous Diffusion Transformer (CoSync-DiT)
  arXiv  Detail & Related papers  (2026-04-14T05:03:57Z)
• OmniForcing: Unleashing Real-time Joint Audio-Visual Generation [51.031726911410594]
  Recent joint audio-visual diffusion models achieve remarkable generation quality but suffer from high latency.<n>We propose OmniForcing, the first framework to distill an offline, dual-stream bidirectional diffusion model into a high-fidelity streaming autore generator.
  arXiv  Detail & Related papers  (2026-03-12T08:17:36Z)
• HiAR: Efficient Autoregressive Long Video Generation via Hierarchical Denoising [52.237486207964245]
  HiAR is a hierarchical denoising framework that reverses the conventional generation order.<n>It performs causal generation across all blocks at every denoising step, so that each block is always conditioned on context at the same noise level.<n>On VBench (20s generation), HiAR achieves the best overall score and the lowest temporal drift among all compared methods.
  arXiv  Detail & Related papers  (2026-03-09T17:58:16Z)
• TempoSyncDiff: Distilled Temporally-Consistent Diffusion for Low-Latency Audio-Driven Talking Head Generation [0.0]
  This paper introduces TempoSyncDiff, a reference-conditioned latent diffusion framework.<n>It explores few-step inference for efficient audio-driven talking-head generation.<n>The framework incorporates identity anchoring and temporal regularization designed to mitigate identity drift and frame-to-frame flicker.
  arXiv  Detail & Related papers  (2026-03-06T09:09:01Z)
• EchoTorrent: Towards Swift, Sustained, and Streaming Multi-Modal Video Generation [8.795438456031512]
  Multi-modal generation models have achieved high visual quality, but their prohibitive latency and limited temporal stability hinder real-time deployment.<n> Streaming inference exacerbates these issues, leading to pronounced multimodal ambiguities, such as blurring, temporal drift, and lip dechronization.<n>We propose EchoTorrent, a novel novel with a fourfold schema: Multi-Teacher Training fine-tunes a pre-trained model on distinct preference domains; Adaptive DMD (ACCDMD) calibrates the audio CFG degradation errors in phased via a schedule; Long Hybrid Tail, which enforces alignment exclusively on tail frames during long-horizon self-roll
  arXiv  Detail & Related papers  (2026-02-14T08:32:38Z)
• Harmony: Harmonizing Audio and Video Generation through Cross-Task Synergy [39.04292189640444]
  Harmony is a novel framework that mechanistically enforces audio-visual synchronization.<n>It establishes a new state-of-the-art, significantly outperforming existing methods in both generation fidelity and, critically, in achieving fine-grained audio-visual synchronization.
  arXiv  Detail & Related papers  (2025-11-26T16:53:05Z)
• Towards One-step Causal Video Generation via Adversarial Self-Distillation [71.30373662465648]
  Recent hybrid video generation models combine autoregressive temporal dynamics with diffusion-based spatial denoising.<n>Our framework produces a single distilled model that flexibly supports multiple inference-step settings.
  arXiv  Detail & Related papers  (2025-11-03T10:12:47Z)
• READ: Real-time and Efficient Asynchronous Diffusion for Audio-driven Talking Head Generation [55.58089937219475]
  We propose READ, the first real-time diffusion-transformer-based talking head generation framework.<n>Our approach first learns highly compressed video latent space via a VAE, significantly reducing the token count to speech generation.<n>We show that READ outperforms state-of-the-art methods by generating competitive talking head videos with significantly reduced runtime.
  arXiv  Detail & Related papers  (2025-08-05T13:57:03Z)
• StPR: Spatiotemporal Preservation and Routing for Exemplar-Free Video Class-Incremental Learning [79.44594332189018]
  Class-Incremental Learning (CIL) seeks to develop models that continuously learn new action categories over time without previously acquired knowledge.<n>Existing approaches either rely on forgetting, raising concerns over memory and privacy, or adapt static image-based methods that neglect temporal modeling.<n>We propose a unified and exemplar-free VCIL framework that explicitly disentangles and preserves information.
  arXiv  Detail & Related papers  (2025-05-20T06:46:51Z)
• When the Future Becomes the Past: Taming Temporal Correspondence for Self-supervised Video Representation Learning [80.09819072780193]
  We propose a self-supervised framework that leverages Temporal Correspondence for video representation learning (T-CoRe)<n>Experiments of T-CoRe consistently present superior performance across several downstream tasks, demonstrating its effectiveness for video representation learning.
  arXiv  Detail & Related papers  (2025-03-19T10:50:03Z)
• ConsistentAvatar: Learning to Diffuse Fully Consistent Talking Head Avatar with Temporal Guidance [27.1886214162329]
  We propose ConsistentAvatar, a novel framework for fully consistent and high-fidelity talking avatar generation. Our method learns to first model the temporal representation for stability between adjacent frames. Extensive experiments demonstrate that ConsistentAvatar outperforms the state-of-the-art methods on the generated appearance, 3D, expression and temporal consistency.
  arXiv  Detail & Related papers  (2024-11-23T03:43:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.