TaoMate: Anchor-Guided Memory Bridging Evolving and Reference States for Real-Time Audio-Video Digital Human Generation

• URL: http://arxiv.org/abs/2607.24359v1
• Date: Mon, 27 Jul 2026 12:35:32 GMT
• Title: TaoMate: Anchor-Guided Memory Bridging Evolving and Reference States for Real-Time Audio-Video Digital Human Generation
• Abstract summary: Real-time long-form digital-human generation relies on causal models to extend audio-visual content.<n>method is an anchor-guided persistent-memory framework for few-step joint audio-video generation.
• Score: 17.429355392513354
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Real-time long-form digital-human generation relies on causal models to extend audio-visual content while preserving subject appearance and audio-video synchronization across successive segments. A bounded cache retains local motion and phonetic context but discards older evidence, whereas attending to the complete generated history is computationally expensive and can propagate accumulated errors. We present \method, an anchor-guided persistent-memory framework for few-step joint audio-video generation. The framework preserves an immutable visual anchor, compresses completed video and audio blocks into fixed-capacity dynamic states, and retrieves those states through modality-specific residual attention without extending the active cache. A reference-aware modulation method additionally conditions video features on dynamic and anchor appearance statistics. Anchor-preserving causal-context distillation varies rollout horizon, prefix provenance, and cache-history reliability while keeping the immutable visual anchor unperturbed. By separating persistent memory from stage-local denoising dependencies, \method further admits stage-parallel execution across blocks, accelerating autoregressive inference without pipeline-specific retraining. We evaluate long-form video continuations with appearance, temporal, synchronization, facial, and speech diagnostics. Results show that \method preserves stable appearance across prompt-conditioned segments and strong audio-visual synchronization under autoregressive generation. Our project page is https://taoliveaigc.github.io/TaoMate.

Related papers

• TetherCache: Stabilizing Autoregressive Long-Form Video Generation with Gated Recall and Trusted Alignment [51.33418612284208]
  We propose TetherCache, a training-free and plug-and-play cache management strategy for drift-resistant long video generation.<n>Gated Recall with Attention-Diversity Balancing selects long-range memory frames using a gated score.<n>TAME lightly edits newly recalled memory tokens by aligning their statistics to a trusted context distribution.
  arXiv  Detail & Related papers  (2026-06-11T08:16:08Z)
• FadeMem: Distance-Aware Memory Consolidation for Autoregressive Video Diffusion [59.207505503284715]
  FadeMem organizes historical KV blocks into a temporal hierarchy under a fixed cache budget.<n>New history is inserted as fine-grained entries, while older adjacent entries are progressively merged.<n>Experiments show improved subject consistency, background stability, and temporal coherence over existing bounded-cache strategies.
  arXiv  Detail & Related papers  (2026-06-09T10:22:18Z)
• Teaching Video Generators to Remember: Eliciting Dynamic Memory for Out-of-Sight State Evolution [25.63670341165374]
  Video models should maintain evolving states when evidence is unobserved, yet current generators often freeze hidden states upon interruption.<n>We introduce ReMind, a framework eliciting dynamic memory behavior via memory-oriented data, event-aware training, and cacheated adaptation.
  arXiv  Detail & Related papers  (2026-05-25T01:30:41Z)
• Echo-Forcing: A Scene Memory Framework for Interactive Long Video Generation [48.476317015122625]
  Echo-Forcing is a training-free scene memory framework for interactive long video generation.<n>It supports smooth transitions, hard cuts, and long-range scene recall under a bounded cache budget.
  arXiv  Detail & Related papers  (2026-05-15T14:33:09Z)
• CoSyncDiT: Cognitive Synchronous Diffusion Transformer for Movie Dubbing [76.74048814837336]
  Movie dubbing aims to synthesize speech that preserves the vocal identity of a reference audio while synchronizing with the lip movements in a target video.<n>Existing methods fail to achieve precise lip-sync and lack naturalness due to explicit alignment at the duration level.<n>We propose a novel flow matching-based movie dubbing framework driven by the Cognitive Synchronous Diffusion Transformer (CoSync-DiT)
  arXiv  Detail & Related papers  (2026-04-14T05:03:57Z)
• Screen, Match, and Cache: A Training-Free Causality-Consistent Reference Frame Framework for Human Animation [44.20260674331104]
  FrameCache is a training-free three-stage framework consisting of Screen, Cache, and Match.<n>Experiments on standard benchmarks demonstrate that FrameCache consistently improves temporal coherence and visual stability.
  arXiv  Detail & Related papers  (2025-12-13T08:45:03Z)
• Mixture of Contexts for Long Video Generation [72.96361488755986]
  We recast long-context video generation as an internal information retrieval task.<n>We propose a simple, learnable sparse attention routing module, Mixture of Contexts (MoC), as an effective long-term memory retrieval engine.<n>As we scale the data and gradually sparsify the routing, the model allocates compute to salient history, preserving identities, actions, and scenes over minutes of content.
  arXiv  Detail & Related papers  (2025-08-28T17:57:55Z)
• InfiniteTalk: Audio-driven Video Generation for Sparse-Frame Video Dubbing [66.48064661467781]
  We introduce sparse-frame video dubbing, a novel paradigm that strategically preserves references to maintain identity, iconic gestures, and camera trajectories.<n>We propose InfiniteTalk, a streaming audio-driven generator designed for infinite-length long sequence dubbing.<n> Comprehensive evaluations on HDTF, CelebV-HQ, and EMTD datasets demonstrate state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-08-19T17:55:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.