Is Forward Prediction Enough? Physical State Grounding for JEPA World Models

• URL: http://arxiv.org/abs/2608.06799v1
• Date: Fri, 07 Aug 2026 04:44:16 GMT
• Title: Is Forward Prediction Enough? Physical State Grounding for JEPA World Models
• Abstract summary: PSG-JEPA shapes its latent space with two complementary grounding objectives beyond forward prediction.<n>We conduct experiments at three levels: (1) latent identifiability via probing, (2) goal-conditioned planning on frozen latents, and (3) policy learning in simulation and on a real robot.
• Score: 18.174985055477666
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Learning structured and control-relevant latent representations remains a key challenge for world models. Recent JEPA-based world models learn action-conditioned predictive latent dynamics from observation sequences. However, their forward-prediction objectives do not explicitly enforce reliable identifiability of robot-centric physical state from individual latents or state changes from latent pairs, which can limit downstream planning and policy performance. We propose PSG-JEPA, a physically grounded JEPA world model that shapes its latent space with two complementary grounding objectives beyond forward prediction: grounding individual latents in robot proprioceptive state, and grounding latent pairs in multi-horizon joint-angle changes. Both objectives are applied only during training, leaving the inference architecture and computational cost unchanged. To comprehensively evaluate PSG-JEPA, we conduct experiments at three levels: (1) latent identifiability via probing, (2) goal-conditioned planning on frozen latents, and (3) policy learning in simulation and on a real robot. Experiments demonstrate that our PSG-JEPA consistently outperforms state-of-the-art latent world-model baselines at all three levels.

Related papers

• Semigroup-JEPA: Latent Dynamics Consistency for Zero-Shot Physics Generalization [32.65135632960651]
  Joint-Embedding Predictive Architecture (JEPA) world models learn a compact latent representation of the world that supports prediction and planning.<n>We introduce SemiGroup-JEPA, which extends the LeWorldModel framework by supplying the parameter governing the physics to the temporal model.<n>We design dynamical tasks under different gravitational fields that, despite obeying the same physical law, exhibit qualitatively different dynamics.
  arXiv  Detail & Related papers  (2026-09-09T17:08:13Z)
• WA-JEPA: Rethinking the Video JEPA Paradigm for World-Action Modeling in Autonomous Driving [8.510409798128272]
  We present WA-JEPA, a VEPA-native world-action model for autonomous driving planning.<n>WA-JEPA employs hybrid future-masked pre-training, where the model infers future latents from observed context.<n>Results: WA-JEPA reaches 91.7S on NAVSIM, surpassing the strongest end-to-end and world-action baselines.
  arXiv  Detail & Related papers  (2026-08-21T10:54:35Z)
• Orthogonal JEPA: Factorized Predictive States for Latent World Models [95.20158869896393]
  We introduce method, a latent world-modeling framework based on predictive factorization.<n>We show that method can allocate redundant capacity to dominant signals while providing weak or conflicting gradients to less dominant predictive structure.<n>Experiments on controlled vision, single-cell transcriptomics, longitudinal health records, continuous control, and molecular dynamics evaluate representation quality, forecasting, planning, and long-horizon stability.
  arXiv  Detail & Related papers  (2026-08-20T13:59:57Z)
• UniJEPA: A Unified Joint-Embedding Predictive Architecture for Task-Agnostic Visual World Modeling [24.745715007832953]
  Joint-Embedding Predictive Architectures (JEPAs) have emerged as a principled framework for self-supervised learning of world models in compact latent spaces.<n>We present UniJEPA, a unified JEPA that jointly learns photometric prediction (image-level transformations) and temporal prediction (video-level next-state dynamics) in one shared latent space.
  arXiv  Detail & Related papers  (2026-08-07T16:55:58Z)
• Sensorimotor World Models: Perception for Action via Inverse Dynamics [59.121736411156384]
  We introduce a sensorimotor world model trained end-to-end with inverse dynamics regularization.<n>It prevents representation collapse and induces action-aligned representations.<n>It learns compact, interpretable latent spaces and enables competitive planning performance across simple 2D and 3D control tasks.
  arXiv  Detail & Related papers  (2026-06-18T11:25:16Z)
• UWM-JEPA: Predictive World Models That Imagine in Belief Space [0.2864713389096699]
  We introduce the Unitary World Model JEPA, a JEPA world model with a density-matrix latent on a joint system-environment space and a learned unitary predictor.<n>The construction preserves the joint-state spectrum exactly during rollout, so the predictor itself cannot dissipate the represented uncertainty.<n>For JEPA world models to imagine under partial observability, latent geometry and predictor dynamics matter, not frozen context-encoding capacity alone.
  arXiv  Detail & Related papers  (2026-05-25T00:28:51Z)
• Entity-Centric World Models: Interaction-Aware Masking for Causal Video Prediction [0.0]
  Joint Embedding Predictive Architectures (JEPA) have set new benchmarks in semantic classification.<n>We propose Interaction-Aware JEPA (IA-JEPA), which utilizes a self-supervised motion-centric masking strategy.<n>IA-JEPA achieves 14.26% accuracy on causal reasoning tasks, a significant lead over the 3.22% achieved by standard patch-masked baselines.
  arXiv  Detail & Related papers  (2026-05-14T23:10:04Z)
• Earth-o1: A Grid-free Observation-native Atmospheric World Model [81.55899748753434]
  We present Earth-o1, an observation-native atmospheric world model that overcomes structural limitations.<n>By integrating diverse sensor inputs into a unified, grid-free dynamical field, the model autonomously advances the atmospheric state in space and time.<n>We show that this fundamentally distinct paradigm enables direct, real-time forecasting and cross-sensor inference without the overhead of explicit numerical solvers.
  arXiv  Detail & Related papers  (2026-05-07T14:27:48Z)
• Agentic World Modeling: Foundations, Capabilities, Laws, and Beyond [209.35045331678043]
  We introduce a "levels x laws" taxonomy organized along two axes.<n>The first defines three capability levels: L1 Predictor, which learns one-step local transition operators; L2 Simulator, which composes them into multi-step, action-conditioned rollouts that respect domain laws; and L3 Evolver, which autonomously revises its own model when predictions fail against new evidence.<n>We synthesize over 400 works and summarize more than 100 representative systems spanning model-based reinforcement learning, video generation, web and GUI agents, multi-agent social simulation, and AI-driven scientific discovery.
  arXiv  Detail & Related papers  (2026-04-24T17:48:47Z)
• Policy-Guided World Model Planning for Language-Conditioned Visual Navigation [12.836371451772438]
  We present PiJEPA, a two-stage framework that combines the strengths of learned navigation policies with latent world model planning for instruction-conditioned visual navigation.<n> Experiments on real-world navigation tasks demonstrate that PiJEPA significantly outperforms both standalone policy execution and uninformed world model planning.
  arXiv  Detail & Related papers  (2026-03-26T23:47:49Z)
• VGGT-World: Transforming VGGT into an Autoregressive Geometry World Model [59.789011777899965]
  VGGT-World is a geometry world model that side-steps video generation entirely and instead forecasts the temporal evolution of frozen geometry-foundation-model features.<n>We show that VGGT-World significantly outperforms the strongest baselines in depth forecasting while running 3.6-5 times faster with only 0.43B trainable parameters.
  arXiv  Detail & Related papers  (2026-03-13T04:56:43Z)
• Causal World Modeling for Robot Control [56.31803788587547]
  Video world models provide the ability to imagine the near future by understanding the causality between actions and visual dynamics.<n>We introduce LingBot-VA, an autoregressive diffusion framework that learns frame prediction and policy execution simultaneously.<n>We evaluate our model on both simulation benchmarks and real-world scenarios, where it shows significant promise in long-horizon manipulation, data efficiency in post-training, and strong generalizability to novel configurations.
  arXiv  Detail & Related papers  (2026-01-29T17:07:43Z)
• Value-guided action planning with JEPA world models [44.84158001773079]
  Building deep learning models that can reason about their environment requires capturing its underlying dynamics.<n>Joint-Embedded Predictive Architectures (JEPA) provide a promising framework to model such dynamics.<n>We propose an approach to enhance planning with JEPA world models by shaping their representation space.
  arXiv  Detail & Related papers  (2025-12-28T20:17:49Z)
• TD-JEPA: Latent-predictive Representations for Zero-Shot Reinforcement Learning [63.73629127832652]
  We introduce TD-JEPA, which leverages TD-based latent-predictive representations into unsupervised RL.<n> TD-JEPA trains explicit state and task encoders, a policy-conditioned multi-step predictor, and a set of parameterized policies directly in latent space.<n> Empirically, TD-JEPA matches or outperforms state-of-the-art baselines on locomotion, navigation, and manipulation tasks across 13 datasets.
  arXiv  Detail & Related papers  (2025-10-01T10:21:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.