Not All Frames Deserve Full Computation: Accelerating Autoregressive Video Generation via Selective Computation and Predictive Extrapolation

• URL: http://arxiv.org/abs/2604.02979v1
• Date: Fri, 03 Apr 2026 11:34:47 GMT
• Title: Not All Frames Deserve Full Computation: Accelerating Autoregressive Video Generation via Selective Computation and Predictive Extrapolation
• Abstract summary: Autoregressive (AR) video diffusion models enable long-form video generation but remain expensive due to repeated multi-step denoising.<n>We present SCOPE, a training-free framework for efficient AR video diffusion.<n>On MAGI-1 and SkyReels-V2, SCOPE achieves up to 4.73x speedup while maintaining quality comparable to the original output.
• Score: 8.70067126225172
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Autoregressive (AR) video diffusion models enable long-form video generation but remain expensive due to repeated multi-step denoising. Existing training-free acceleration methods rely on binary cache-or-recompute decisions, overlooking intermediate cases where direct reuse is too coarse yet full recomputation is unnecessary. Moreover, asynchronous AR schedules assign different noise levels to co-generated frames, yet existing methods process the entire valid interval uniformly. To address these AR-specific inefficiencies, we present SCOPE, a training-free framework for efficient AR video diffusion. SCOPE introduces a tri-modal scheduler over cache, predict, and recompute, where prediction via noise-level Taylor extrapolation fills the gap between reuse and recomputation with explicit stability controls backed by error propagation analysis. It further introduces selective computation that restricts execution to the active frame interval. On MAGI-1 and SkyReels-V2, SCOPE achieves up to 4.73x speedup while maintaining quality comparable to the original output, outperforming all training-free baselines.

Related papers

• Evolving Cache Schedules for Fast Diffusion Policy Inference [6.879031540914794]
  Diffusion policies achieve strong visuomotor control by iteratively denoising action chunks, but repeated denoising makes real-time deployment computationally demanding.<n>We introduce Evolving Cache Schedules (EVO), a training-free acceleration framework that globally schedules cache refreshes via evolutionary search.<n>EVO preserves near-full performance while substantially reducing computation, achieving up to 8.05x action-generation speedup and reducing FLOPs from 15.77G to as low as 1.96G.
  arXiv  Detail & Related papers  (2026-07-22T15:40:11Z)
• Surprise Forcing: What to Remember, When to Skip in Long Video Generation [65.0022589149937]
  Streaming autoregressive diffusion makes minute-scale video synthesis practical, but its bounded context and fixed denoising schedule allocate resources uniformly.<n>We introduce Surprise Forcing, a training-free framework that treats both limitations as online resource-allocation problems.<n> Experiments on VBench, VBench-Long, and VBench-2.0 show that the proposed allocation strategy improves long-horizon consistency and visual quality while retaining real-time streaming throughput.
  arXiv  Detail & Related papers  (2026-07-20T18:37:58Z)
• Reducing Temporal Redundancy for Efficient Vision-Language-Action Inference [19.683948028719808]
  Vision-Language-Action (VLA) models exhibit strong generalization for robotic manipulation, yet their high inference latency limits real time deployment.<n>We propose a system level acceleration strategy that reduces computation in both perception and action generation.<n> Experiments on Libero, RobotWin, and Real Robot Platforms demonstrate over 2 times speedup while maintaining high performance.
  arXiv  Detail & Related papers  (2026-07-14T02:48:31Z)
• RhymeFlow: Training-Free Acceleration for Video Generation with Asynchronous Denoising Flow Scheduling [51.279397568734424]
  Video generation models based on Diffusion Transformers (DiTs) have achieved remarkable performance in video synthesis.<n>DiTs suffer from high inference latency and computational costs due to the quadratic complexity of 3D attention.<n>We introduce bftextRhymeFlow, a training-free framework that decouples the denoising trajectories of different frames.
  arXiv  Detail & Related papers  (2026-06-04T15:49:37Z)
• SoulX-LiveAct: Towards Hour-Scale Real-Time Human Animation with Neighbor Forcing and ConvKV Memory [25.57144961436966]
  Autoregressive (AR) diffusion models offer a promising framework for sequential generation tasks such as video synthesis.<n>We propose Neighbor Forcing, a diffusion-step-consistent AR formulation that propagates temporally adjacent frames as latent neighbors under the same noise condition.<n>Our approach significantly improves training convergence, hour-scale generation quality, and inference efficiency compared to existing AR diffusion methods.
  arXiv  Detail & Related papers  (2026-03-12T09:49:58Z)
• USV: Unified Sparsification for Accelerating Video Diffusion Models [11.011602744993942]
  Unified Sparsification for Video diffusion models is an end-to-end trainable framework.<n>It orchestrates sparsification across both the model's internal computation and its sampling process.<n>It achieves up to 83.3% speedup in the denoising process and 22.7% end-to-end acceleration, while maintaining high visual fidelity.
  arXiv  Detail & Related papers  (2025-12-05T14:40:06Z)
• OmniSAT: Compact Action Token, Faster Auto Regression [70.70037017501357]
  We introduce an Omni Swift Action Tokenizer, which learns a compact, transferable action representation.<n>The resulting discrete tokenization shortens the training sequence by 6.8$times$, and lowers the target entropy.
  arXiv  Detail & Related papers  (2025-10-08T03:55:24Z)
• Sortblock: Similarity-Aware Feature Reuse for Diffusion Model [9.749736545966694]
  Diffusion Transformers (DiTs) have demonstrated remarkable generative capabilities.<n>DiTs' sequential denoising process results in high inference latency.<n>We propose Sortblock, a training-free inference acceleration framework.
  arXiv  Detail & Related papers  (2025-08-01T08:10:54Z)
• Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion [67.94300151774085]
  We introduce Self Forcing, a novel training paradigm for autoregressive video diffusion models.<n>It addresses the longstanding issue of exposure bias, where models trained on ground-truth context must generate sequences conditioned on their own imperfect outputs.
  arXiv  Detail & Related papers  (2025-06-09T17:59:55Z)
• Rethinking Video Tokenization: A Conditioned Diffusion-based Approach [58.164354605550194]
  New tokenizer, Diffusion Conditioned-based Gene Tokenizer, replaces GAN-based decoder with conditional diffusion model.<n>We trained using only a basic MSE diffusion loss for reconstruction, along with KL term and LPIPS perceptual loss from scratch.<n>Even a scaled-down version of CDT (3$times inference speedup) still performs comparably with top baselines.
  arXiv  Detail & Related papers  (2025-03-05T17:59:19Z)
• Faster Image2Video Generation: A Closer Look at CLIP Image Embedding's Impact on Spatio-Temporal Cross-Attentions [27.111140222002653]
  This paper investigates the role of CLIP image embeddings within the Stable Video Diffusion (SVD) framework. We introduce the VCUT, a training-free approach optimized for efficiency within the SVD architecture. The implementation of VCUT leads to a reduction of up to 322T Multiple-Accumulate Operations (MACs) per video and a decrease in model parameters by up to 50M, achieving a 20% reduction in latency compared to the baseline.
  arXiv  Detail & Related papers  (2024-07-27T08:21:14Z)
• Cross-Consistent Deep Unfolding Network for Adaptive All-In-One Video Restoration [78.14941737723501]
  We propose a Cross-consistent Deep Unfolding Network (CDUN) for All-In-One VR. By orchestrating two cascading procedures, CDUN achieves adaptive processing for diverse degradations. In addition, we introduce a window-based inter-frame fusion strategy to utilize information from more adjacent frames.
  arXiv  Detail & Related papers  (2023-09-04T14:18:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.