LaWM: Least Action World Models for Long-Horizon Physical Consistency from Visual Observations

• URL: http://arxiv.org/abs/2605.08279v1
• Date: Fri, 08 May 2026 07:03:13 GMT
• Title: LaWM: Least Action World Models for Long-Horizon Physical Consistency from Visual Observations
• Abstract summary: We propose Least Action World Models (LaWM) to operationalize the Principle of Least Action in learned visual latent space.<n>LaWM encodes observations into learned generalized coordinates, learns a latent discrete Lagrangian over consecutive latent states.<n>Because the transition is induced by a discrete variational principle, LaWM provides a structure-preserving bias for long-horizon visual prediction.
• Score: 9.35652906982887
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Learning predictive world models from visual observations is a core problem in embodied AI, with applications to model-based reinforcement learning and robotic planning. Existing latent world models typically generate future states with unconstrained neural transition functions, while modern video generation systems often prioritize perceptual plausibility or introduce physical structure through auxiliary losses, external guidance, or separate dynamics modules. As a result, long-horizon rollouts can remain weakly grounded in the physical principles that govern real dynamics, leading to compounding error, energy drift, and physically inconsistent futures. We propose Least Action World Models (LaWM), a latent world-modeling framework that operationalizes the Principle of Least Action in learned visual latent space: future rollouts are governed by a learned Lagrangian action functional rather than produced only by an unconstrained transition predictor. Our main technical realization is a latent variational integrator: LaWM encodes observations into learned generalized coordinates, learns a latent discrete Lagrangian over consecutive latent states, constructs a discrete action functional, and advances prediction by solving the corresponding discrete integration condition. Thus, physical structure is not merely used to score, regularize, or constrain a completed trajectory; it defines the latent transition rule itself. Because the transition is induced by a discrete variational principle, LaWM provides a structure-preserving bias for long-horizon visual prediction. Across physics-clean synthetic dynamics and embodied robot interaction benchmarks, LaWM improves physical invariance, background consistency, motion smoothness, and appearance and geometric prediction metrics over video-generation and world-model baselines.

Related papers

• Semigroup-JEPA: Latent Dynamics Consistency for Zero-Shot Physics Generalization [32.65135632960651]
  Joint-Embedding Predictive Architecture (JEPA) world models learn a compact latent representation of the world that supports prediction and planning.<n>We introduce SemiGroup-JEPA, which extends the LeWorldModel framework by supplying the parameter governing the physics to the temporal model.<n>We design dynamical tasks under different gravitational fields that, despite obeying the same physical law, exhibit qualitatively different dynamics.
  arXiv  Detail & Related papers  (2026-09-09T17:08:13Z)
• DyG$^2$T: Modeling Object Dynamics with 3D Gaussian Temporal-Spatial Particle Graph Transformer [86.4618122245946]
  DyG$2$T is a dynamics modeling framework that infers object motion trajectories by spatially completing and temporally discriminating Key Point representations.<n>Experiments on both synthetic and real-world datasets demonstrate that DyG$2$T achieves accurate dynamics modeling and reasoning.
  arXiv  Detail & Related papers  (2026-08-19T03:44:28Z)
• Sensorimotor World Models: Perception for Action via Inverse Dynamics [59.121736411156384]
  We introduce a sensorimotor world model trained end-to-end with inverse dynamics regularization.<n>It prevents representation collapse and induces action-aligned representations.<n>It learns compact, interpretable latent spaces and enables competitive planning performance across simple 2D and 3D control tasks.
  arXiv  Detail & Related papers  (2026-06-18T11:25:16Z)
• ACWM-Phys: Investigating Generalized Physical Interaction in Action-Conditioned Video World Models [15.357110248749214]
  Action-conditioned world models (ACWMs) have shown strong promise for video prediction and decision-making.<n>We introduce ACWM-Phys, a new benchmark for evaluating action-conditioned prediction under diverse physical dynamics.
  arXiv  Detail & Related papers  (2026-05-09T00:00:47Z)
• The Global Neural World Model: Spatially Grounded Discrete Topologies for Action-Conditioned Planning [0.9137554315375919]
  We present the Global Neural World Model (GNWM), a self-stabilizing framework that achieves topological quantization.<n> operating as a continuous, action-conditioned Joint-Embedding Predictive Architecture (JEPA)<n>Our results show this architecture prevents manifold drift during autoregressive rollouts by using grid snapping'' as a native error-correction mechanism.
  arXiv  Detail & Related papers  (2026-04-17T15:12:15Z)
• ResWM: Residual-Action World Model for Visual RL [0.06999740786886534]
  Learning predictive world models from raw visual observations is a central challenge in reinforcement learning (RL)<n>We introduce the Residual-Action World Model (ResWM), a new framework that reformulates the control variable from absolute actions to residual actions.<n>ResWM produces more stable and energy-efficient action trajectories, a property critical for robotic systems deployed in real-world environments.
  arXiv  Detail & Related papers  (2026-03-11T11:27:08Z)
• Walk through Paintings: Egocentric World Models from Internet Priors [65.30611174953958]
  We present the Egocentric World Model (EgoWM), a simple, architecture-agnostic method that transforms any pretrained video diffusion model into an action-conditioned world model.<n>Rather than training from scratch, we repurpose the rich world priors of Internet-scale video models and inject motor commands through lightweight conditioning layers.<n>Our approach scales naturally across embodiments and action spaces, ranging from 3-DoF mobile robots to 25-DoF humanoids.
  arXiv  Detail & Related papers  (2026-01-21T18:59:32Z)
• Aligning Agentic World Models via Knowledgeable Experience Learning [68.85843641222186]
  We introduce WorldMind, a framework that constructs a symbolic World Knowledge Repository by synthesizing environmental feedback.<n>WorldMind achieves superior performance compared to baselines with remarkable cross-model and cross-environment transferability.
  arXiv  Detail & Related papers  (2026-01-19T17:33:31Z)
• Semantic Belief-State World Model for 3D Human Motion Prediction [0.0]
  We propose a Semantic Belief-State World Model that reframes human motion prediction as latent dynamical simulation on the human body manifold.<n>Inspired by belief-state world models developed for model-based reinforcement learning, SBWM adapts latent transitions and rollout-centric training to the domain of human motion.
  arXiv  Detail & Related papers  (2026-01-07T02:06:26Z)
• Prismatic World Model: Learning Compositional Dynamics for Planning in Hybrid Systems [38.4555621948915]
  Prismatic World Model (PRISM-WM) is designed to decompose complex hybrid dynamics into composable primitives.<n>PRISM-WM significantly reduces rollout drift by accurately modeling sharp mode transitions in system dynamics.
  arXiv  Detail & Related papers  (2025-12-09T09:40:34Z)
• Learning Primitive Embodied World Models: Towards Scalable Robotic Learning [50.32986780156215]
  We propose a novel paradigm for world modeling--Primitive Embodied World Models (PEWM)<n>By restricting video generation to fixed short horizons, our approach enables fine-grained alignment between linguistic concepts and visual representations of robotic actions.<n>Our framework bridges the gap between fine-grained physical interaction and high-level reasoning, paving the way toward scalable, interpretable, and general-purpose embodied intelligence.
  arXiv  Detail & Related papers  (2025-08-28T14:31:48Z)
• Latent Space Energy-based Neural ODEs [73.01344439786524]
  This paper introduces novel deep dynamical models designed to represent continuous-time sequences.<n>We train the model using maximum likelihood estimation with Markov chain Monte Carlo.<n> Experimental results on oscillating systems, videos and real-world state sequences (MuJoCo) demonstrate that our model with the learnable energy-based prior outperforms existing counterparts.
  arXiv  Detail & Related papers  (2024-09-05T18:14:22Z)
• SEGNO: Generalizing Equivariant Graph Neural Networks with Physical Inductive Biases [66.61789780666727]
  We show how the second-order continuity can be incorporated into GNNs while maintaining the equivariant property. We also offer theoretical insights into SEGNO, highlighting that it can learn a unique trajectory between adjacent states. Our model yields a significant improvement over the state-of-the-art baselines.
  arXiv  Detail & Related papers  (2023-08-25T07:15:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.