A Generalization Theory for JEPA-Based World Models

• URL: http://arxiv.org/abs/2606.27014v1
• Date: Thu, 25 Jun 2026 13:29:20 GMT
• Title: A Generalization Theory for JEPA-Based World Models
• Abstract summary: Joint Embedding Predictive Architectures (JEPAs) have emerged as a promising paradigm for world modeling.<n>Despite their empirical success, the theoretical understanding of JEPA-based world models remains limited.
• Score: 34.09833992943572
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Joint Embedding Predictive Architectures (JEPAs) have recently emerged as a promising paradigm for world modeling by learning predictive dynamics in a latent space rather than generating future observations at the input level. Despite their empirical success, the theoretical understanding of JEPA-based world models remains limited. In this paper, we develop the first generalization theory for JEPA-based world models. We formulate JEPA pretraining as a conditional spectral graph learning problem and show that the JEPA objective is equivalent to a low-rank factorization of an action-conditioned co-occurrence matrix. Building on this characterization, we establish a connection between JEPA pretraining error and downstream planning regret, leading to a finite-sample generalization bound for JEPA-based world models. Our analysis reveals an inherent trade-off between approximation and sample errors with respect to the latent dimension, providing theoretical insights into the advantages and limitations of latent predictive models compared with input-level predictive approaches.

Related papers

• UniJEPA: A Unified Joint-Embedding Predictive Architecture for Task-Agnostic Visual World Modeling [24.745715007832953]
  Joint-Embedding Predictive Architectures (JEPAs) have emerged as a principled framework for self-supervised learning of world models in compact latent spaces.<n>We present UniJEPA, a unified JEPA that jointly learns photometric prediction (image-level transformations) and temporal prediction (video-level next-state dynamics) in one shared latent space.
  arXiv  Detail & Related papers  (2026-08-07T16:55:58Z)
• Beyond Generative Priors: Minority Sampling with JEPA-Guided Diffusion [7.276372008305617]
  We propose a world-centric perspective on minority sampling, which defines rarity with respect to real-world priors.<n>We introduce JEPA guidance, a diffusion sampling framework guided by a Joint-Embedding Predictive Architecture.<n>We show that JEPA guidance consistently improves the fidelity and semantic validity of minority samples.
  arXiv  Detail & Related papers  (2026-05-23T15:40:56Z)
• World Action Models: The Next Frontier in Embodied AI [123.5787299299832]
  Vision-Language-Action (VLA) models have achieved strong semantic generalization for embodied policy learning.<n>They learn reactive observation-to-action mappings without explicitly modeling how the physical world evolves under intervention.<n>A growing body of work addresses this limitation by integrating world models, predictive models of environment dynamics, into the action generation pipeline.
  arXiv  Detail & Related papers  (2026-05-12T13:10:52Z)
• Sub-JEPA: Subspace Gaussian Regularization for Stable End-to-End World Models [26.928611870426383]
  JointEmbedding Predictive Architectures (JEPAs) provide a simple framework for learning world models by predicting future latent representations.<n>The recent LeWorldModel (LeWM) shows that this issue can be alleviated bysimply constraining latent embeddings with an isotropic Gaussian prior.<n>We propose ame, which seeks a favorable operatingpoint on the bias-variance frontier by applying Gaussian constraints inmultiple random subspaces rather than in the originalembedding space.
  arXiv  Detail & Related papers  (2026-05-10T00:51:47Z)
• Self-Supervised JEPA-based World Models for LiDAR Occupancy Completion and Forecasting [11.278785857643575]
  We propose textbfAD-LiST-JEPA, a self-supervised world model for autonomous driving that predicts futuretemporal evolution from LiDAR data.<n>We evaluate the quality of the learned representations through a downstream-based occupancy completion and forecasting task.
  arXiv  Detail & Related papers  (2026-02-13T02:42:21Z)
• VJEPA: Variational Joint Embedding Predictive Architectures as Probabilistic World Models [0.0]
  We introduce emphVariational JEPA (VJEPA), a textitprobabilistic generalization that learns a predictive distribution over future latent states via a variational objective.<n>VJEPA representations can serve as sufficient information states for optimal control without pixel reconstruction, while providing formal guarantees for collapse avoidance.<n>We propose emphBayesian JEPA (BJEPA), an extension that factorizes the predictive belief into a learned dynamics expert and a modular prior expert.
  arXiv  Detail & Related papers  (2026-01-20T18:04:16Z)
• Value-guided action planning with JEPA world models [44.84158001773079]
  Building deep learning models that can reason about their environment requires capturing its underlying dynamics.<n>Joint-Embedded Predictive Architectures (JEPA) provide a promising framework to model such dynamics.<n>We propose an approach to enhance planning with JEPA world models by shaping their representation space.
  arXiv  Detail & Related papers  (2025-12-28T20:17:49Z)
• LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristics [53.247652209132376]
  Joint-Embedding Predictive Architectures (JEPAs) offer a promising blueprint, but lack of practical guidance and theory has led to ad-hoc R&D.<n>We present a comprehensive theory of JEPAs and instantiate it in bf LeJEPA, a lean, scalable, and theoretically grounded training objective.
  arXiv  Detail & Related papers  (2025-11-11T18:21:55Z)
• Identifying and Mitigating Social Bias Knowledge in Language Models [52.52955281662332]
  We propose a novel debiasing approach, Fairness Stamp (FAST), which enables fine-grained calibration of individual social biases.<n>FAST surpasses state-of-the-art baselines with superior debiasing performance.<n>This highlights the potential of fine-grained debiasing strategies to achieve fairness in large language models.
  arXiv  Detail & Related papers  (2024-08-07T17:14:58Z)
• A PAC-Bayesian Perspective on the Interpolating Information Criterion [54.548058449535155]
  We show how a PAC-Bayes bound is obtained for a general class of models, characterizing factors which influence performance in the interpolating regime. We quantify how the test error for overparameterized models achieving effectively zero training error depends on the quality of the implicit regularization imposed by e.g. the combination of model, parameter-initialization scheme.
  arXiv  Detail & Related papers  (2023-11-13T01:48:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.