No Gaussian Required: Contrastive Inverse Dynamics for JEPA World Models

• URL: http://arxiv.org/abs/2608.17542v1
• Date: Tue, 18 Aug 2026 09:03:35 GMT
• Title: No Gaussian Required: Contrastive Inverse Dynamics for JEPA World Models
• Abstract summary: Joint-Embedding Predictive Architectures (JEPAs) learn world models by predicting future embeddings.<n>Every practical system adds an anti-collapse mechanism.<n>We argue that the anti-collapse pressure can instead come from the transition data itself.
• Score: 2.506624215459612
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Joint-Embedding Predictive Architectures (JEPAs) learn world models by predicting future embeddings, but the objective admits a trivial solution of a constant encoder, so every practical system adds an anti-collapse mechanism (LeCun, 2022; Assran et al., 2023; Bardes et al., 2022; 2024). LeWorldModel (LeWM) prevents collapse with SIGReg, a regularizer that forces the latent distribution to match an isotropic Gaussian: the representation is stabilized by prescribing what it must look like, independently of the environment it models. We argue that the anti-collapse pressure can instead come from the transition data itself. Action-Contrastive Masked Transition Modeling (AC-MTM) keeps LeWM's forward latent-prediction objective and adds a training-only inverse-dynamics head trained with Action-NCE: each latent transition must identify the action that produced it among the other actions in the batch, a discrimination task that a collapsed encoder provably fails. The inverse branch is discarded after training, leaving test-time encoding, forward prediction, planning, and compute identical to LeWM. On four standard pixel-control tasks under a matched planning protocol, AC-MTM trains stably from scratch and matches SIGReg on average. On the harder multi-object OGBench Visual Scene task, results are consistent with the prescribed geometry becoming a bottleneck: AC-MTM reaches 80.0$\pm$2.0% success versus 58.0$\pm$2.0% for SIGReg, improving by 20-24 points in each training seed. A single 50-episode random-policy run gives a 52% baseline estimate. Contrastive inverse dynamics thus provides a distribution-free anti-collapse signal that requires no target network, stop-gradient, pretrained encoder, or reconstruction objective, and we characterize the action-space and observability assumptions under which it holds. We make our code available at https://github.com/jackboyla/action-contrastive-jepa

Related papers

• Schema-Adaptive Action-Conditioned JEPA for Cross-Machine CNC Transfer under Partial Sensor Overlap [0.0]
  Cross-machine deployment of industrial world models requires transfer across changes in dynamics, sensing interfaces, sampling regimes, and control units.<n>We study a schema-adaptive action-conditioned Joint-Embedding Predictive Architecture (SAAC-JEPA) for CNC dynamics.<n> Evaluation uses group-disjoint source splits, source-only normalization, held-out self-supervised validation, unit audits, and a sealed target test after model locking.
  arXiv  Detail & Related papers  (2026-09-13T15:10:06Z)
• GaussianDream++: Efficient 3D Gaussian World Modeling for Robotic Manipulation [55.47130289185285]
  Vision-Language-Action (VLA) policies have advanced language-conditioned robotic manipulation.<n>We present textbfMethodname, a compact, policy-native extension that inserts textbfWorld State Tokens and textbfWorld Prediction Tokens directly into the VLA backbone.
  arXiv  Detail & Related papers  (2026-08-26T11:41:55Z)
• Sensorimotor World Models: Perception for Action via Inverse Dynamics [59.121736411156384]
  We introduce a sensorimotor world model trained end-to-end with inverse dynamics regularization.<n>It prevents representation collapse and induces action-aligned representations.<n>It learns compact, interpretable latent spaces and enables competitive planning performance across simple 2D and 3D control tasks.
  arXiv  Detail & Related papers  (2026-06-18T11:25:16Z)
• Diffusion Transformer World-Action Model for AV Scene Prediction [0.0]
  Action-conditioned world models let an autonomous vehicle predict future camera scenes from its own planned controls.<n>But at compact, trainable scale the futures are ambiguous and the field's standard distortion metrics actively mislead.<n>We confront this with a compact latent world model that predicts future scene latents a frozen decoder renders to $256 times 256$ frames up to 8 seconds ahead.
  arXiv  Detail & Related papers  (2026-06-11T07:24:12Z)
• UWM-JEPA: Predictive World Models That Imagine in Belief Space [0.2864713389096699]
  We introduce the Unitary World Model JEPA, a JEPA world model with a density-matrix latent on a joint system-environment space and a learned unitary predictor.<n>The construction preserves the joint-state spectrum exactly during rollout, so the predictor itself cannot dissipate the represented uncertainty.<n>For JEPA world models to imagine under partial observability, latent geometry and predictor dynamics matter, not frozen context-encoding capacity alone.
  arXiv  Detail & Related papers  (2026-05-25T00:28:51Z)
• Grounded World Model for Semantically Generalizable Planning [94.53923128709965]
  We learn a Grounded World Model (GWM) in a vision-language-aligned latent space.<n>Each proposed action is scored based on how close its future outcome is to the task instruction.<n>On the proposed WISER benchmark, GWM-MPC achieves a 87% success rate on the test set.
  arXiv  Detail & Related papers  (2026-04-13T17:25:41Z)
• Label-Free Cross-Task LoRA Merging with Null-Space Compression [50.63908869296697]
  We introduce Null-Space Compression (NSC) Merging, a label-free, output-agnostic method that sets merge weights from adapter geometry.<n>NSC achieves state-of-the-art performance across twenty heterogeneous vision tasks with balanced gains where prior methods overfit subsets of tasks.
  arXiv  Detail & Related papers  (2026-03-27T11:34:41Z)
• LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels [49.35636088613484]
  Joint Embedding Predictive Architectures (JEPAs) offer a compelling framework for learning world models in compact latent spaces.<n>We introduce LeModelWorld, the first JEPA that trains stably end-to-end from raw pixels using only two loss terms.<n>With 15M parameters trainable on a single GPU in a few hours, LeWM plans up to 48x faster than foundation-model-based world models.
  arXiv  Detail & Related papers  (2026-03-13T19:48:14Z)
• Beyond Reward Suppression: Reshaping Steganographic Communication Protocols in MARL via Dynamic Representational Circuit Breaking [0.0]
  steganographic collusion, where agents develop private protocols to evade monitoring, presents a critical AI safety threat.<n>Existing defenses, limited to behavioral or reward layers, fail to detect coordination in latent communication channels.<n>We introduce the Dynamic Representational Circuit Breaker (DRCB), an architectural defense operating at the optimization substrate.
  arXiv  Detail & Related papers  (2026-03-07T04:14:38Z)
• Weak-SIGReg: Covariance Regularization for Stable Deep Learning [0.0]
  This work adopts Sketched Isotropic Gaussian Regularization ( SIGReg), recently introduced in the LeJEPA self-supervised framework, and repurposes it as a general optimization stabilizer for supervised learning.<n>Inspired by interacting particle systems, representation collapse is viewed as drift; SIGReg constrains the representation density towards an isotropic Gaussian, mitigating this drift.<n> Empirically, SIGReg recovers the training of a ViT on CIFAR-100 from a collapsed 20.73% to 72.02% accuracy without architectural hacks and significantly improves the convergence of deep vanillas trained with pure SGD. Code is available at
  arXiv  Detail & Related papers  (2026-03-06T05:32:40Z)
• Multistep Inverse Is Not All You Need [87.62730694973696]
  In real-world control settings, the observation space is often unnecessarily high-dimensional and subject to time-correlated noise. It is therefore desirable to learn an encoder to map the observation space to a simpler space of control-relevant variables. We propose a new algorithm, ACDF, which combines multistep-inverse prediction with a latent forward model.
  arXiv  Detail & Related papers  (2024-03-18T16:36:01Z)
• Consistent-Teacher: Towards Reducing Inconsistent Pseudo-targets in Semi-supervised Object Detection [28.40887130075552]
  Pseudo-targets undermine the training of an accurate detector. It injects noise into the student's training, leading to severe overfitting problems. We propose a systematic solution, termed ConsistentTeacher, to reduce the inconsistency.
  arXiv  Detail & Related papers  (2022-09-04T10:21:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.