HeadCast: Casting Attention Heads for Efficient Autoregressive Video Generation

• URL: http://arxiv.org/abs/2607.20125v1
• Date: Wed, 22 Jul 2026 13:29:35 GMT
• Title: HeadCast: Casting Attention Heads for Efficient Autoregressive Video Generation
• Abstract summary: HeadCast is a training-free, plug-and-play acceleration framework built on the observation that a pre-trained AR model's attention heads exhibit stable, heterogeneous behaviors.<n>HeadCast accelerates inference by up to 1.62x at 720P and 1.95x at 1080P, while keeping VBench quality on par with full attention and largely flicker-free.
• Score: 7.896241308705432
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Autoregressive (AR) video diffusion models have become a promising paradigm for long and streaming video synthesis, but the continuously growing Key-Value (KV) cache makes attention the dominant inference cost, especially at high resolution where each frame contributes many tokens. Existing remedies either evict the cache with coarse heuristics that cause inter-frame flickering, or require model re-training. We propose HeadCast, a training-free, plug-and-play acceleration framework built on the observation that a pre-trained AR model's attention heads exhibit stable, heterogeneous behaviors. After a short warm-up, HeadCast performs a one-time classification at the maximum-noise step that sorts every head into one of four archetypes: Sink, Dummy, Spatial, and Global, and restructures the monolithic KV cache into head-specific pathways. Crucially, it retains the Global heads that preserve the long-range temporal consistency aggressive eviction destroys. Because the Spatial pathway operates on a fixed-size grid, its savings grow with resolution: across state-of-the-art AR models, HeadCast accelerates inference by up to 1.62x at 720P and 1.95x at 1080P, while keeping VBench quality on par with full attention and largely flicker-free. Code is available at https://github.com/sjlgaga/HeadCast .

Related papers

• HASTE: Training-Free Video Diffusion Acceleration via Head-Wise Adaptive Sparse Attention [68.95533683996236]
  Training-free sparse attention is attractive because it accelerates pretrained models without retraining.<n>Existing online top-$p$ sparse attention still spends non-negligible cost on mask prediction.<n>We show that these two overlooked factors limit the practical speed-quality trade-off of training-free sparse attention in Video DiTs.
  arXiv  Detail & Related papers  (2026-05-14T07:57:55Z)
• Head Forcing: Long Autoregressive Video Generation via Head Heterogeneity [21.347969441374506]
  Autoregressive video diffusion models support real-time synthesis but suffer from error accumulation and context loss over long horizons.<n>We propose Head Forcing, a training-free framework that assigns each head type a tailored KV cache strategy.<n>Without additional training, Head Forcing extends generation from 5 seconds to minute-level duration, supports multi-prompt interactive synthesis, and consistently outperforms existing baselines.
  arXiv  Detail & Related papers  (2026-05-14T07:27:39Z)
• Pyramid Forcing: Head-Aware Pyramid KV Cache Policy for High-Quality Long Video Generation [14.413680197991356]
  Pyramid Forcing is a head-aware KVCache framework that identifies head types offline and assigns behavior-specific cache policies.<n> Experiments show that Pyramid Forcing consistently improves long-horizon generation quality on VBench-Long.
  arXiv  Detail & Related papers  (2026-05-13T07:23:02Z)
• Forcing-KV: Hybrid KV Cache Compression for Efficient Autoregressive Video Diffusion Models [32.39747481484621]
  We introduce KV cache compression into autoregressive video diffusion.<n>We propose Forcing-KV, a hybrid KV cache compression strategy that performs structured static pruning for static heads and dynamic pruning based on segment-wise similarity for dynamic heads.<n>Our method achieves a generation speed of over 29 frames per second on a single NVIDIA H200 GPU along with 30% cache memory reduction, delivering up to 1.35x and 1.50x speedups on LongLive and Self Forcing at 480P resolution, and further scaling to 2.82x speedup at 1080P resolution.
  arXiv  Detail & Related papers  (2026-05-10T17:59:21Z)
• Motion-Aware Caching for Efficient Autoregressive Video Generation [73.27762884730272]
  MotionCache exploits inter-frame differences as a lightweight proxy for pixel-level motion characteristics.<n>Experiments on state-of-the-art models like SkyReels-V2 and MAGI-1 demonstrate that MotionCache achieves significant speedups.
  arXiv  Detail & Related papers  (2026-05-03T05:49:27Z)
• Fast Autoregressive Video Diffusion and World Models with Temporal Cache Compression and Sparse Attention [37.91838955436801]
  Autoregressive video diffusion models enable streaming generation, opening the door to long-form synthesis, video world models, and interactive neural game engines.<n>As generation progresses, the KV cache grows, causing both increasing latency and escalating GPU memory, which in turn restricts usable temporal context and harms long-range consistency.<n>We propose a unified, training-free attention framework for autoregressive diffusion: TempCache compresses the KV cache via temporal correspondence to bound cache growth; AnnCA accelerates cross-attention by selecting frame-relevant prompt tokens using fast approximate nearest neighbor matching; and AnnSA sparsifies self-attention by restricting each query
  arXiv  Detail & Related papers  (2026-02-02T08:31:21Z)
• Efficient Autoregressive Video Diffusion with Dummy Head [35.72735205078644]
  We propose Dummy Forcing, a method to control context accessibility across different heads.<n>Specifically, the proposed heterogeneous memory allocation reduces head-wise context redundancy, accompanied by dynamic head programming.<n>Without additional training, our Dummy Forcing delivers up to 2.0x speedup over the baseline, supporting video generation at 24.3 FPS with less than 0.5% quality drop.
  arXiv  Detail & Related papers  (2026-01-28T11:20:43Z)
• Towards Holistic Modeling for Video Frame Interpolation with Auto-regressive Diffusion Transformers [95.68243351895107]
  We propose a holistic, video-centric paradigm named textbfLocal textbfDiffusion textbfForcing for textbfVideo textbfFrame textbfInterpolation (LDF-VFI)<n>Our framework is built upon an auto-regressive diffusion transformer that models the entire video sequence to ensure long-range temporal coherence.<n>LDF-VFI achieves state-of-the-art performance on challenging long-sequence benchmarks, demonstrating superior per
  arXiv  Detail & Related papers  (2026-01-21T12:58:52Z)
• PackCache: A Training-Free Acceleration Method for Unified Autoregressive Video Generation via Compact KV-Cache [61.57938553036056]
  We introduce PackCache, a training-free KV-cache management method that compacts the KV cache through three coordinated mechanisms.<n>In terms of efficiency, PackCache accelerates end-to-end generation by 1.7-2.2x on 48-frame long sequences.
  arXiv  Detail & Related papers  (2026-01-07T19:51:06Z)
• Next Tokens Denoising for Speech Synthesis [51.320443764269726]
  Dragon-FM is a novel text-to-speech (TTS) design that unifies AR and flow-matching.<n>It processes 48 kHz audio tokens in chunks at a compact rate of 12.5 tokens per second.<n>Experiments on podcast datasets demonstrate its capability to efficiently generate high-quality zero-shot podcasts.
  arXiv  Detail & Related papers  (2025-07-30T15:03:36Z)
• Training-Free Efficient Video Generation via Dynamic Token Carving [54.52061549312799]
  Jenga is an inference pipeline that combines dynamic attention carving with progressive resolution generation.<n>As a plug-and-play solution, Jenga enables practical, high-quality video generation on modern hardware.
  arXiv  Detail & Related papers  (2025-05-22T16:21:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.