Token Radius Attention for Efficient Video Generation

• URL: http://arxiv.org/abs/2608.02504v1
• Date: Mon, 03 Aug 2026 17:05:15 GMT
• Title: Token Radius Attention for Efficient Video Generation
• Abstract summary: We propose a training-free framework that maps query entropy to an analytic token budget and converts it into a temporally decayed radius without explicit key ranking.<n>Across seven Wan2.1, Wan2.2, and HunyuanVideo T2V/I2V configurations, TRA retains only 9-19% of attention interactions and achieves 1.56x-2.05x speedup with competitive generation quality.
• Score: 14.657468653848627
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Video Diffusion Transformers (VDiTs) enable high-fidelity generation but incur quadratic cost from dense 3D self-attention. Existing head- and block-level sparse methods share computation budgets across queries, overlooking token-specific attention demand. We observe that retained density varies across queries yet correlates log-linearly with attention entropy, while dominant interactions form query-centered neighborhoods with token-dependent radii. Based on these findings, we propose Token Radius Attention (TRA), a training-free framework that maps query entropy to an analytic token budget and converts it into a temporally decayed radius without explicit key ranking. Fused entropy extraction, warm-up reuse, and block-sparse mask construction further reduce overhead. Across seven Wan2.1, Wan2.2, and HunyuanVideo T2V/I2V configurations, TRA retains only 9-19% of attention interactions and achieves 1.56x-2.05x speedup with competitive generation quality. Code is available at https://github.com/IF-LAB-PKU/Token-Radius-Attention.

Related papers

• SPADE: An Input-Adaptive Sparse Attention Engine for Fast Video Diffusion Models Inference [7.309663528163573]
  Video diffusion transformers (vDiTs) generate high quality but pay quadratic self-attention cost.<n>We present SPADE, a training-free sparse-attention engine of three parts.<n>Across Hunyuan-Video and Wan 2.1/2.2 for text-to-video and image-to-video generation, SPADE raises sparsity and speed while preserving quality.
  arXiv  Detail & Related papers  (2026-08-04T08:47:57Z)
• Chimera: Designing and Chinchilla-Scaling Hybrid Visual Diffusion Transformers [54.01042826416009]
  Visual generation requires high-resolution images, long videos, and multimodal context.<n>We introduce Chimera, a hybrid visual diffusion backbone with a principled scaling recipe.<n>We show that the dense backbone is 1.7x as compute-efficient as a matched full-attention Wan2.1 2B baseline.
  arXiv  Detail & Related papers  (2026-07-30T17:58:02Z)
• Surprise Forcing: What to Remember, When to Skip in Long Video Generation [65.0022589149937]
  Streaming autoregressive diffusion makes minute-scale video synthesis practical, but its bounded context and fixed denoising schedule allocate resources uniformly.<n>We introduce Surprise Forcing, a training-free framework that treats both limitations as online resource-allocation problems.<n> Experiments on VBench, VBench-Long, and VBench-2.0 show that the proposed allocation strategy improves long-horizon consistency and visual quality while retaining real-time streaming throughput.
  arXiv  Detail & Related papers  (2026-07-20T18:37:58Z)
• V-CAST: Video Curvature-Aware Spatio-Temporal Pruning for Efficient Video Large Language Models [48.80617385008755]
  Video language models (VideoLLMs) show strong capability in understanding, yet long-context inference is still dominated by massive redundant visual tokens in the prefill stage.<n>We propose V-CAST, a training-free, plug-and-play pruning policy for long-context video inference.
  arXiv  Detail & Related papers  (2026-03-29T11:53:32Z)
• Going Down Memory Lane: Scaling Tokens for Video Stream Understanding with Dynamic KV-Cache Memory [50.30283773196725]
  Existing approaches rely on key-value caching to accumulate frame-level details over time, but use a limited number of tokens per frame.<n>We propose scaling the token budget to enable more granular-temporal understanding and reasoning.
  arXiv  Detail & Related papers  (2026-02-20T18:59:50Z)
• Light Forcing: Accelerating Autoregressive Video Diffusion via Sparse Attention [28.598033369607723]
  textscLight Forcing is a textitfirst sparse attention solution tailored for AR video generation models.<n>It incorporates a textitChunk-Aware Growth mechanism to quantitatively estimate the contribution of each chunk.<n>We also introduce a textit Sparse Attention to capture informative historical and local context in a coarse-to-fine manner.
  arXiv  Detail & Related papers  (2026-02-04T17:41:53Z)
• PISA: Piecewise Sparse Attention Is Wiser for Efficient Diffusion Transformers [37.401543107035046]
  Diffusion Transformers are fundamental for video and image generation, but their efficiency is bottlenecked by the quadratic complexity of attention.<n>We propose PISA, a training-free Piecewise Sparse Attention that covers the full attention span with sub-quadratic complexity.
  arXiv  Detail & Related papers  (2026-02-01T07:47:06Z)
• Dense Video Understanding with Gated Residual Tokenization [49.17263029080152]
  High temporal resolution is essential for capturing fine-grained details in video understanding.<n>Current benchmarks rely mostly on low-frame-rate sampling.<n>Dense Video Understanding (DVU) enables high-FPS video comprehension by reducing both tokenization time and token overhead.
  arXiv  Detail & Related papers  (2025-09-17T17:34:40Z)
• Sparse-vDiT: Unleashing the Power of Sparse Attention to Accelerate Video Diffusion Transformers [24.105473321347894]
  We propose Sparse-vDiT, a sparsity acceleration framework for Video Diffusion Transformer (vDiT)<n>We show that Sparse-vDiT achieves 2.09$times$, 2.38$times$, and 1.67$times$ theoretical FLOP reduction, and actual inference speedups of 1.76$times$, 1.85$times$, and 1.58$times$, respectively.<n>Our work demonstrates that latent structural sparsity in vDiTs can be systematically exploited for long video synthesis.
  arXiv  Detail & Related papers  (2025-06-03T16:42:37Z)
• Sparse VideoGen2: Accelerate Video Generation with Sparse Attention via Semantic-Aware Permutation [84.00166854547241]
  Diffusion Transformers (DiTs) are essential for video generation but suffer from significant latency due to the quadratic complexity of attention.<n>We propose SVG2, a training-free framework that maximizes identification accuracy and computation minimizes waste.
  arXiv  Detail & Related papers  (2025-05-24T21:30:29Z)
• VSA: Faster Video Diffusion with Trainable Sparse Attention [38.37291040904089]
  Scaling video diffusion transformers (DiTs) is limited by their quadratic 3D attention, even though most of the attention mass concentrates on a small subset of positions.<n>We turn this observation into VSA, a trainable, hardware-efficient sparse attention that replaces full attention at emphboth training and inference.
  arXiv  Detail & Related papers  (2025-05-19T17:30:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.