Mitigating Compounding Error via Video Representation Regularization

• URL: http://arxiv.org/abs/2607.27036v1
• Date: Wed, 29 Jul 2026 15:29:39 GMT
• Title: Mitigating Compounding Error via Video Representation Regularization
• Abstract summary: Video diffusion-based world models enable long autoregressive video generation for robotics, autonomous driving and simulation tasks.<n>sliding-window autoregressive inference suffers from severe error accumulation that degrades frame quality over time.<n>We investigate the internal representation dynamics of video world models and discover that compounding error is tightly coupled with dimensional collapse of hidden representations.<n>We propose video representation regularization, a lightweight training constraint that stabilizes latent representations and suppresses iterative error accumulation.
• Score: 32.14962855907007
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Video diffusion-based world models enable long autoregressive video generation for robotics, autonomous driving and simulation tasks, yet sliding-window autoregressive inference suffers from severe error accumulation that degrades frame quality over time. Although this phenomenon has been widely observed, the underlying mechanism of compounding error and how to achieve stable long-horizon generation remain largely unresolved. In this paper, we investigate the internal representation dynamics of video world models and discover that compounding error is tightly coupled with dimensional collapse of hidden representations. Specifically, the effective rank of model representations sharply decreases at the onset of generation drift, revealing a strong connection between representational degradation and long-term rollout instability. Furthermore, we find that pure training data scaling fails to boost model resistance to error drift, contradicting mainstream scaling paradigms. To address this problem, we propose video representation regularization, a lightweight training constraint that stabilizes latent representations and suppresses iterative error accumulation. Compared with Diffusion Forcing, our method achieves improvements from 38.65 to 55.56 and from 44.37 to 72.08 on the Aesthetic Quality and Imaging Quality metrics of VBench. Our work establishes the first connection between autoregressive video drifting and model internal representations, adopts erank as a quantitative metric for error accumulation, reveals counterintuitive scaling limitations for video world models, and presents a simple yet effective regularization strategy to improve long video generation robustness.

Related papers

• Cycle-World: Mitigating Error Accumulation in Long-term Video World Models via Reverse-Prediction Cycle Consistency [123.12549538776109]
  Cycle-World is a framework designed for stable and temporally consistent long-video generation.<n>Our approach tackles error drift by enforcing strict temporal reversibility across both the training and inference phases.<n>Experiments on the VBench benchmark demonstrate that Cycle-World's dual-phase synergy significantly mitigates error drift.
  arXiv  Detail & Related papers  (2026-07-13T17:27:13Z)
• Flex-Forcing: Towards a Unified Autoregressive and Bidirectional Video Diffusion Model [88.42070387375679]
  We introduce Flex-Forcing, a unified training and inference framework that enables a video diffusion model to seamlessly operate under both bidirectional and autoregressive generation regimes.<n>Experiments on multiple video generation benchmarks demonstrate that Flex-Forcing achieves consistently better video quality, long-video stability than strong baselines with a rigid inference schedule.
  arXiv  Detail & Related papers  (2026-07-03T17:34:14Z)
• Towards Error-Free Long Video Generation [56.86952045212838]
  In this paper, we introduce an infinite-length video generation framework that produces high-quality, dynamic, and identity-consistent single-shot long videos.<n>We first finetune a diffusion model as a video extension model on large-scale short video data to autoregressively generate temporally coherent clips.<n>Our framework establishes a new benchmark for realistic and coherent minute-level video synthesis.
  arXiv  Detail & Related papers  (2026-06-21T07:39:37Z)
• Accelerating Training of Autoregressive Video Generation Models via Local Optimization with Representation Continuity [57.83511884904928]
  We explore methods to accelerate training for autoregressive video generation models through empirical analyses.<n>Our results reveal that while training on fewer video frames significantly reduces training time, it also exacerbates error accumulation and introduces inconsistencies in the generated videos.<n>Inspired by Lipschitz continuity, we propose a Representation Continuity (ReCo) strategy to improve the consistency of generated videos.
  arXiv  Detail & Related papers  (2026-04-08T09:43:03Z)
• Adaptive Video Distillation: Mitigating Oversaturation and Temporal Collapse in Few-Step Generation [43.89162138967428]
  We propose a novel distillation framework tailored specifically for video diffusion models.<n>Its core innovations include: (1) an adaptive regression loss that dynamically adjusts spatial supervision weights to prevent artifacts arising from excessive distribution shifts; (2) a temporal regularization loss that promotes smooth and physically plausible sampling trajectories; and (3) an inference-time frame strategy that reduces sampling overhead while preserving perceptual quality.<n>Experiments and ablation studies on the VBench and VBench2 benchmarks demonstrate that our method achieves stable few-step video synthesis, significantly enhancing perceptual fidelity and motion realism.
  arXiv  Detail & Related papers  (2026-03-23T11:54:33Z)
• LoL: Longer than Longer, Scaling Video Generation to Hour [50.945885467651216]
  This work achieves the first demonstration of real-time, streaming, and infinite-length video generation with little quality decay.<n>As an illustration, we generate continuous videos up to 12 hours in length, which, to our knowledge, is among the longest publicly demonstrated results in streaming video generation.
  arXiv  Detail & Related papers  (2026-01-23T17:21:35Z)
• VideoAR: Autoregressive Video Generation via Next-Frame & Scale Prediction [31.191310873846177]
  VideoAR is the first large-scale Visual Autoregressive framework for video generation that combines multi-scale next-frame prediction with autoregressive modeling.<n>VideoAR disentangles spatial and temporal dependencies by integrating intra-frame VAR with causal next-frame prediction, supported by a 3D multi-scale tokenizer.<n> Empirically, VideoAR achieves new state-of-the-art results improving resolutions among autoregressive models, FVD on UCF-101 from 99.5 to 88.6 while reducing inference steps over 10x, and reaching a VBench score of 81.74-competitive with diffusion-based
  arXiv  Detail & Related papers  (2026-01-09T17:34:59Z)
• Pack and Force Your Memory: Long-form and Consistent Video Generation [26.53691150499802]
  Long-form video generation presents a dual challenge.<n>Models must capture long-range dependencies while preventing the error accumulation inherent in autoregressive decoding.<n>MemoryPack and Direct Forcing substantially enhance the context consistency and reliability of long-form video generation.
  arXiv  Detail & Related papers  (2025-10-02T08:22:46Z)
• Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion [67.94300151774085]
  We introduce Self Forcing, a novel training paradigm for autoregressive video diffusion models.<n>It addresses the longstanding issue of exposure bias, where models trained on ground-truth context must generate sequences conditioned on their own imperfect outputs.
  arXiv  Detail & Related papers  (2025-06-09T17:59:55Z)
• Unsupervised Video Decomposition using Spatio-temporal Iterative Inference [31.97227651679233]
  Multi-object scene decomposition is a fast-emerging problem in learning. We show that our model has a high accuracy even without color information. We demonstrate the decomposition, segmentation prediction capabilities of our model and show that it outperforms the state-of-the-art on several benchmark datasets.
  arXiv  Detail & Related papers  (2020-06-25T22:57:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.