Energy-Structured Latent World Models with Neural Time Fields for Physically Constistent Open-World Motion Planning

• URL: http://arxiv.org/abs/2608.09876v1
• Date: Mon, 10 Aug 2026 17:31:18 GMT
• Title: Energy-Structured Latent World Models with Neural Time Fields for Physically Constistent Open-World Motion Planning
• Abstract summary: We propose a novel Energy-Structured Latent World Model (ELWM)<n>Our key idea is to structure the ELWM latent state to explicitly carry energy and momentum, ensuring strictly causal transitions.<n>We further implement this for motion planning by constructing Physics-Conditioned Neural Time Fields (PC-NTF)<n>Compared to generic latent models, PC-NTF reduces 0.8-s motion-prediction NRMSE from 0.36 to 0.29.
• Score: 25.402498594599013
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Physically consistent motion planning remains a fundamental challenge in embodied AI, as generated trajectories must strictly conform to real-world execution dynamics. While latent world models offer a promising approach by predicting these dynamics, existing methods learn unconstrained future representations where absorbed physics remains implicit. Therefore, they fail to form reusable physical knowledge, which compromises reliability in unpredictable open-world navigation. To address this, we propose a novel Energy-Structured Latent World Model (ELWM). Our key idea is to structure the ELWM latent state to explicitly carry energy and momentum, ensuring strictly causal transitions via dissipation and control ports. Trained on multimodal RGB-D and inertial interaction histories, our model guarantees physically consistent predictions. We further implement this for motion planning by constructing Physics-Conditioned Neural Time Fields (PC-NTF), a key technical cornerstone that integrates ELWM into an arrival time field via the Eikonal equation to yield a physically-informed navigation policy. Across held-out scenes, our evaluation reveals significant improvements. Compared to generic latent models, PC-NTF reduces 0.8-s motion-prediction NRMSE from 0.36 to 0.29. Against Active Neural Time Fields, it improves navigation success from 81.3% to 89.7% and SPL from 0.64 to 0.73, while cutting the physical collision rate from 12.1% to 5.8% and the Eikonal residual from 0.083 to 0.031. Beyond these targeted gains, our results demonstrate that embedding explicit physical structures into latent spaces intrinsically bridges the gap between predictive world models and safe, dynamically feasible motion planning.

Related papers

• Semigroup-JEPA: Latent Dynamics Consistency for Zero-Shot Physics Generalization [32.65135632960651]
  Joint-Embedding Predictive Architecture (JEPA) world models learn a compact latent representation of the world that supports prediction and planning.<n>We introduce SemiGroup-JEPA, which extends the LeWorldModel framework by supplying the parameter governing the physics to the temporal model.<n>We design dynamical tasks under different gravitational fields that, despite obeying the same physical law, exhibit qualitatively different dynamics.
  arXiv  Detail & Related papers  (2026-09-09T17:08:13Z)
• PhyLatent: Learning Dynamics-Relevant Representations for JEPA World Models [12.761667671058767]
  PhyLatent is a dynamics-relevant training objective for JointEmbedding Predictive Architecture (JEPA) world models.<n>We identify three failure modes in JEPA world models: physical invariance collapse, physical identifiability collapse, and counterfactual dynamics collapse.<n>On OGBench-Cube, PhyLatent reduces the three failure rates from 15.60%, 6.71%, and 8.41% to 7.53%, 0.95%, and 4.62%, respectively, and improves model predictive control (MPC) success from 70.0% to 78.1%.
  arXiv  Detail & Related papers  (2026-08-06T08:02:15Z)
• A Physics-Grounded Benchmark for Multi-Agent Dynamics in World Models [96.64934195520802]
  We introduce CrashTwin, a physics-grounded evaluation framework designed to stress-test the physical trustworthiness of world models.<n>CrashTwin couples a diverse dataset of multi-agent collision scenarios, comprising 25K controllable synthetic 12K in-the-wild real-world collision sequences.<n>We propose a diagnostic suite that systematically evaluates three dimensions:-temporal consistency, momentum and kinetic energy conservation, and world-dynamics integrity.
  arXiv  Detail & Related papers  (2026-06-27T06:13:35Z)
• Sensorimotor World Models: Perception for Action via Inverse Dynamics [59.121736411156384]
  We introduce a sensorimotor world model trained end-to-end with inverse dynamics regularization.<n>It prevents representation collapse and induces action-aligned representations.<n>It learns compact, interpretable latent spaces and enables competitive planning performance across simple 2D and 3D control tasks.
  arXiv  Detail & Related papers  (2026-06-18T11:25:16Z)
• PH-Dreamer: A Physics-Driven World Model via Port-Hamiltonian Generative Dynamics [0.0]
  World models built on recurrent state space architectures produce dynamics that violate conservation and dissipative principles.<n>We introduce a unified Port-Hamiltonian framework that remedies this through three synergistic mechanisms.<n>Across visual control benchmarks, this paradigm not only attains superior returns but also elevates internal fidelity by establishing a tighter, lower variance alignment between imagined and real rewards, all while reducing latent phase space volume by 4.18-8.41%, energy consumption by up to 7.80%, and mean squared jerk by up to 9.38%.
  arXiv  Detail & Related papers  (2026-05-18T12:20:54Z)
• LaWM: Least Action World Models for Long-Horizon Physical Consistency from Visual Observations [9.35652906982887]
  We propose Least Action World Models (LaWM) to operationalize the Principle of Least Action in learned visual latent space.<n>LaWM encodes observations into learned generalized coordinates, learns a latent discrete Lagrangian over consecutive latent states.<n>Because the transition is induced by a discrete variational principle, LaWM provides a structure-preserving bias for long-horizon visual prediction.
  arXiv  Detail & Related papers  (2026-05-08T07:03:13Z)
• CtrlAttack: A Unified Attack on World-Model Control in Diffusion Models [92.04182855442254]
  We analyze the vulnerability of I2V models, find that temporal control mechanisms constitute a new attack surface, and reveal the challenge of modeling them uniformly.<n>We propose a trajectory-control attack, called CtrlAttack, to interfere with state evolution during the generation process.<n> Experimental results show that even under low-dimensional and strongly regularized perturbation constraints, our method can still significantly disrupt temporal consistency.
  arXiv  Detail & Related papers  (2026-03-13T08:05:50Z)
• When World Models Dream Wrong: Physical-Conditioned Adversarial Attacks against World Models [54.08784776767683]
  We present Physical-Conditioned World Model Attack (PhysCond-WMA), the first white-box world model attack that perturbs physical-condition channels.<n>PhysCond-WMA induces semantic, logic, or decision-level distortion while preserving perceptual fidelity.
  arXiv  Detail & Related papers  (2026-02-21T07:22:37Z)
• Reactive Motion Generation With Particle-Based Perception in Dynamic Environments [14.960949776319147]
  Reactive motion generation in dynamic and unstructured scenarios is typically subject to essentially static perception and system dynamics.<n>This article focuses on revealing tight connection between reactive planning and dynamic mapping for manipulators from a model-based perspective.<n>We propose an obstacle-aware MPPI-based planning formulation that jointly propagates robot-obstacle dynamics.
  arXiv  Detail & Related papers  (2026-02-18T13:48:54Z)
• Aligning Agentic World Models via Knowledgeable Experience Learning [68.85843641222186]
  We introduce WorldMind, a framework that constructs a symbolic World Knowledge Repository by synthesizing environmental feedback.<n>WorldMind achieves superior performance compared to baselines with remarkable cross-model and cross-environment transferability.
  arXiv  Detail & Related papers  (2026-01-19T17:33:31Z)
• Forecasting Continuous Non-Conservative Dynamical Systems in SO(3) [51.510040541600176]
  We propose a novel approach to modeling the rotation of moving objects in computer vision.<n>Our approach is agnostic to energy and momentum conservation while being robust to input noise.<n>By learning to approximate object dynamics from noisy states during training, our model attains robust extrapolation capabilities in simulation and various real-world settings.
  arXiv  Detail & Related papers  (2025-08-11T09:03:10Z)
• PI-WAN: A Physics-Informed Wind-Adaptive Network for Quadrotor Dynamics Prediction in Unknown Environments [3.4802474792943805]
  We introduce the Physics-Informed Wind-Adaptive Network (PI-WAN), which embeds physical constraints directly into the training process for robust quadrotor dynamics learning.<n>Specifically, PI-WAN employs a Temporal Convolutional Network (TCN) architecture that efficiently captures temporal dependencies from historical flight data.<n>By incorporating real-time prediction results into a model predictive control (MPC) framework, we achieve improvements in closed-loop tracking performance.
  arXiv  Detail & Related papers  (2025-07-01T14:48:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.