A Mechanistic View on Video Generation as World Models: State and Dynamics

• URL: http://arxiv.org/abs/2601.17067v1
• Date: Thu, 22 Jan 2026 19:00:18 GMT
• Title: A Mechanistic View on Video Generation as World Models: State and Dynamics
• Abstract summary: This work proposes a novel taxonomy centered on two pillars: State Construction and Dynamics Modeling.<n>By addressing these challenges, the field can evolve from generating visually plausible videos to building robust, general-purpose world simulators.
• Score: 43.951972667861575
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large-scale video generation models have demonstrated emergent physical coherence, positioning them as potential world models. However, a gap remains between contemporary "stateless" video architectures and classic state-centric world model theories. This work bridges this gap by proposing a novel taxonomy centered on two pillars: State Construction and Dynamics Modeling. We categorize state construction into implicit paradigms (context management) and explicit paradigms (latent compression), while dynamics modeling is analyzed through knowledge integration and architectural reformulation. Furthermore, we advocate for a transition in evaluation from visual fidelity to functional benchmarks, testing physical persistence and causal reasoning. We conclude by identifying two critical frontiers: enhancing persistence via data-driven memory and compressed fidelity, and advancing causality through latent factor decoupling and reasoning-prior integration. By addressing these challenges, the field can evolve from generating visually plausible videos to building robust, general-purpose world simulators.

Related papers

• Platonic Representation Hypothesis on World Models [42.05154017234555]
  We show that capable world models evolve toward geometrically similar internal structures.<n>Our findings suggest that the pursuit of predictive consistency can promote shared, transition-compatible latent structure across world models.
  arXiv  Detail & Related papers  (2026-08-24T18:08:30Z)
• World Action Models: The Next Frontier in Embodied AI [123.5787299299832]
  Vision-Language-Action (VLA) models have achieved strong semantic generalization for embodied policy learning.<n>They learn reactive observation-to-action mappings without explicitly modeling how the physical world evolves under intervention.<n>A growing body of work addresses this limitation by integrating world models, predictive models of environment dynamics, into the action generation pipeline.
  arXiv  Detail & Related papers  (2026-05-12T13:10:52Z)
• Feed-Forward 3D Scene Modeling: A Problem-Driven Perspective [91.23306722968509]
  Generalizable feed-forward 3D reconstruction has witnessed rapid development in recent years.<n>Existing feed-forward approaches share similar high-level architectural patterns.<n>We propose a novel taxonomy centered on model design strategies that are agnostic to the output format.
  arXiv  Detail & Related papers  (2026-04-15T16:07:18Z)
• The Trinity of Consistency as a Defining Principle for General World Models [106.16462830681452]
  General World Models are capable of learning, simulating, and reasoning about objective physical laws.<n>We propose a principled theoretical framework that defines the essential properties requisite for a General World Model.<n>Our work establishes a principled pathway toward general world models, clarifying both the limitations of current systems and the architectural requirements for future progress.
  arXiv  Detail & Related papers  (2026-02-26T16:15:55Z)
• RISE-Video: Can Video Generators Decode Implicit World Rules? [71.92434352963427]
  We present RISE-Video, a pioneering reasoning-oriented benchmark for Text-Image-to-Video (TI2V) synthesis.<n>RISE-Video comprises 467 meticulously human-annotated samples spanning eight rigorous categories.<n>We propose an automated pipeline leveraging Large Multimodal Models (LMMs) to emulate human-centric assessment.
  arXiv  Detail & Related papers  (2026-02-05T18:36:10Z)
• Simulating the Visual World with Artificial Intelligence: A Roadmap [48.64639618440864]
  Video generation is shifting from generating visually appealing clips to building virtual environments that support interaction and maintain physical plausibility.<n>This survey provides a systematic overview of this evolution, conceptualizing modern video foundation models as the combination of two core components.<n>We trace the progression of video generation through four generations, culminating in a video generation model that embodies intrinsic physical plausibility.
  arXiv  Detail & Related papers  (2025-11-11T18:59:50Z)
• PhysCorr: Dual-Reward DPO for Physics-Constrained Text-to-Video Generation with Automated Preference Selection [10.498184571108995]
  We propose PhysCorr, a unified framework for modeling, evaluating, and optimizing physical consistency in video generation.<n>Specifically, we introduce PhysicsRM, the first dual-dimensional reward model that quantifies both intra-object stability and inter-object interactions.<n>Our approach is model-agnostic and scalable, enabling seamless integration into a wide range of video diffusion and transformer-based backbones.
  arXiv  Detail & Related papers  (2025-11-06T02:40:57Z)
• Are Video Models Ready as Zero-Shot Reasoners? An Empirical Study with the MME-CoF Benchmark [124.00111584020834]
  We conduct an empirical study to investigate whether video models are ready to serve as zero-shot reasoners.<n>We focus on the leading and popular Veo-3.<n>We evaluate its reasoning behavior across 12 dimensions, including spatial, geometric, physical, temporal, and embodied logic.
  arXiv  Detail & Related papers  (2025-10-30T17:59:55Z)
• Can World Models Benefit VLMs for World Dynamics? [59.73433292793044]
  We investigate the capabilities when world model priors are transferred into Vision-Language Models.<n>We name our best-performing variant Dynamic Vision Aligner (DyVA)<n>We find DyVA to surpass both open-source and proprietary baselines, achieving state-of-the-art or comparable performance.
  arXiv  Detail & Related papers  (2025-10-01T13:07:05Z)
• Dyn-O: Building Structured World Models with Object-Centric Representations [42.65409148846005]
  We introduce Dyn-O, an enhanced structured world model built upon object-centric representations.<n>Compared to prior work in object-centric representations, Dyn-O improves in both learning representations and modeling dynamics.<n>We find that our method can learn object-centric world models directly from pixel observations, outperforming DreamerV3 in rollout prediction accuracy.
  arXiv  Detail & Related papers  (2025-07-04T05:06:15Z)
• Unified Multimodal Understanding and Generation Models: Advances, Challenges, and Opportunities [22.476740954286836]
  We present a comprehensive survey aimed at guiding future research.<n>We review existing unified models, categorizing them into three main architectural paradigms.<n>We discuss the key challenges facing this nascent field, including tokenization strategy, cross-modal attention, and data.
  arXiv  Detail & Related papers  (2025-05-05T11:18:03Z)
• Aether: Geometric-Aware Unified World Modeling [49.33579903601599]
  Aether is a unified framework that enables geometry-aware reasoning in world models.<n>Our framework achieves zero-shot generalization in both action following and reconstruction tasks.<n>We hope our work inspires the community to explore new frontiers in physically-reasonable world modeling.
  arXiv  Detail & Related papers  (2025-03-24T17:59:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.