Streaming Autoregressive Video Generation via Diagonal Distillation

• URL: http://arxiv.org/abs/2603.09488v2
• Date: Wed, 11 Mar 2026 14:49:12 GMT
• Title: Streaming Autoregressive Video Generation via Diagonal Distillation
• Abstract summary: Autoregressive models offer a natural framework for sequential frame synthesis but require heavy computation to achieve high fidelity.<n>We propose Diagonal Distillation to exploit temporal information across both video chunks and denoising steps.<n>Our method generates a 5-second video in 2.61 seconds (up to 31 FPS), achieving a 277.3x speedup over the undistilled model.
• Score: 50.13573884115673
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large pretrained diffusion models have significantly enhanced the quality of generated videos, and yet their use in real-time streaming remains limited. Autoregressive models offer a natural framework for sequential frame synthesis but require heavy computation to achieve high fidelity. Diffusion distillation can compress these models into efficient few-step variants, but existing video distillation approaches largely adapt image-specific methods that neglect temporal dependencies. These techniques often excel in image generation but underperform in video synthesis, exhibiting reduced motion coherence, error accumulation over long sequences, and a latency-quality trade-off. We identify two factors that result in these limitations: insufficient utilization of temporal context during step reduction and implicit prediction of subsequent noise levels in next-chunk prediction (i.e., exposure bias). To address these issues, we propose Diagonal Distillation, which operates orthogonally to existing approaches and better exploits temporal information across both video chunks and denoising steps. Central to our approach is an asymmetric generation strategy: more steps early, fewer steps later. This design allows later chunks to inherit rich appearance information from thoroughly processed early chunks, while using partially denoised chunks as conditional inputs for subsequent synthesis. By aligning the implicit prediction of subsequent noise levels during chunk generation with the actual inference conditions, our approach mitigates error propagation and reduces oversaturation in long-range sequences. We further incorporate implicit optical flow modeling to preserve motion quality under strict step constraints. Our method generates a 5-second video in 2.61 seconds (up to 31 FPS), achieving a 277.3x speedup over the undistilled model.

Related papers

• Towards Error-Free Long Video Generation [56.86952045212838]
  In this paper, we introduce an infinite-length video generation framework that produces high-quality, dynamic, and identity-consistent single-shot long videos.<n>We first finetune a diffusion model as a video extension model on large-scale short video data to autoregressively generate temporally coherent clips.<n>Our framework establishes a new benchmark for realistic and coherent minute-level video synthesis.
  arXiv  Detail & Related papers  (2026-06-21T07:39:37Z)
• FlowLong: Inference-time Long Video Generation via Manifold-constrained Tweedie Matching [68.01498128172214]
  We propose a novel but simple inference-time approach for long video generation that is architecture-agnostic and requires no additional training.<n>Our method generates long videos via overlapping sliding windows, where predicted clean samples from adjacent windows are blended via emphTweedie matching to enforce both textbfmanifold constraint and temporal consistency across overlap regions.<n>Our method generates videos several times longer than the native window length while outperforming both training-free and autoregressive baselines in temporal consistency and visual quality, and further extends to audio-video joint generation and text-to-3DGS without any
  arXiv  Detail & Related papers  (2026-05-20T08:55:37Z)
• Adaptive Video Distillation: Mitigating Oversaturation and Temporal Collapse in Few-Step Generation [43.89162138967428]
  We propose a novel distillation framework tailored specifically for video diffusion models.<n>Its core innovations include: (1) an adaptive regression loss that dynamically adjusts spatial supervision weights to prevent artifacts arising from excessive distribution shifts; (2) a temporal regularization loss that promotes smooth and physically plausible sampling trajectories; and (3) an inference-time frame strategy that reduces sampling overhead while preserving perceptual quality.<n>Experiments and ablation studies on the VBench and VBench2 benchmarks demonstrate that our method achieves stable few-step video synthesis, significantly enhancing perceptual fidelity and motion realism.
  arXiv  Detail & Related papers  (2026-03-23T11:54:33Z)
• JoyAvatar: Real-time and Infinite Audio-Driven Avatar Generation with Autoregressive Diffusion [19.420963062956222]
  JoyAvatar is an audio-driven autoregressive model capable of real-time inference and infinite-length video generation.<n>Our model achieves competitive results in visual quality, temporal consistency, and lip synchronization.
  arXiv  Detail & Related papers  (2025-12-12T10:06:01Z)
• Rolling Forcing: Autoregressive Long Video Diffusion in Real Time [86.40480237741609]
  Rolling Forcing is a novel video generation technique that enables streaming long videos with minimal error accumulation.<n>Rolling Forcing comes with three novel designs. First, instead of iteratively sampling individual frames, which accelerates error propagation, we design a joint denoising scheme.<n>Second, we introduce the attention sink mechanism into the long-horizon stream video generation task, which allows the model to keep key value states of initial frames as a global context anchor.<n>Third, we design an efficient training algorithm that enables few-step distillation over largely extended denoising windows.
  arXiv  Detail & Related papers  (2025-09-29T17:57:14Z)
• POSE: Phased One-Step Adversarial Equilibrium for Video Diffusion Models [18.761042377485367]
  POSE (Phased One-Step Equilibrium) is a distillation framework that reduces the sampling steps of large-scale video diffusion models.<n>We show that POSE outperforms other acceleration methods on VBench-I2V by average 7.15% in semantic alignment, temporal conference and frame quality.
  arXiv  Detail & Related papers  (2025-08-28T17:20:01Z)
• SwiftVideo: A Unified Framework for Few-Step Video Generation through Trajectory-Distribution Alignment [76.60024640625478]
  Diffusion-based or flow-based models have achieved significant progress in video synthesis but require multiple iterative sampling steps.<n>We propose a unified and stable distillation framework that combines the advantages of trajectory-preserving and distribution-matching strategies.<n>Our method maintains high-quality video generation while substantially reducing the number of inference steps.
  arXiv  Detail & Related papers  (2025-08-08T07:26:34Z)
• Fast constrained sampling in pre-trained diffusion models [80.99262780028015]
  We propose an algorithm that enables fast, high-quality generation under arbitrary constraints.<n>Our approach produces results that rival or surpass the state-of-the-art training-free inference methods.
  arXiv  Detail & Related papers  (2024-10-24T14:52:38Z)
• Training-Free Adaptive Diffusion with Bounded Difference Approximation Strategy [44.09909260046396]
  We propose AdaptiveDiffusion to reduce noise prediction steps during the denoising process. Our method can significantly speed up the denoising process while generating identical results to the original process, achieving up to an average 25x speedup.
  arXiv  Detail & Related papers  (2024-10-13T15:19:18Z)
• Identifying and Solving Conditional Image Leakage in Image-to-Video Diffusion Model [31.70050311326183]
  Diffusion models tend to generate videos with less motion than expected. We address this issue from both inference and training aspects. Our methods outperform baselines by producing higher motion scores with lower errors.
  arXiv  Detail & Related papers  (2024-06-22T04:56:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.