When Does LeJEPA Learn a World Model?

• URL: http://arxiv.org/abs/2605.26379v1
• Date: Mon, 25 May 2026 22:56:26 GMT
• Title: When Does LeJEPA Learn a World Model?
• Abstract summary: A representation that scrambles the true degrees of freedom of the world cannot support reliable planning or compositional generalization.<n>We prove that LeJEPA linearly recovers the world's latent variables from nonlinear observations, a property known as linear identifiability.<n>We validate the theory with experiments ranging from 2D examples to 1024-dimensional latents, including distributional ablations and pixel-based robotic control.
• Score: 48.74247491501631
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A representation that scrambles the true degrees of freedom of the world cannot support reliable planning or compositional generalization. We prove that LeJEPA (alignment plus Gaussian regularization) linearly recovers the world's latent variables from nonlinear observations, a property known as linear identifiability, in a broad class of worlds where latents evolve under stationary, additive-noise transitions. Our main result is that among all such worlds, the Gaussian is the unique latent distribution for which this guarantee holds. The forward direction rests on a spectral decomposition in which each degree of nonlinearity is strictly penalized by alignment, making the linear map the optimum; the converse rules out every non-Gaussian alternative. We further prove an approximate identifiability result where the guarantee degrades gracefully, and show that linear, orthogonal identifiability enables optimal latent-space planning. We validate the theory with experiments ranging from 2D examples to 1024-dimensional latents, including distributional ablations and pixel-based robotic control. Our theory turns an empirically successful recipe into a mathematical guarantee, providing the foundation for building World Models that provably recover the structure of the world.

Related papers

• Identifiability Without Gaussianity: Symbolic World Models and Near-Infinite Temporal Consistency [0.0]
  We prove that statistical World Models cannot achieve near-infinite temporal consistency for any non-Gaussian system.<n>In non-Gaussian regimes, the only condition for near-infinite temporal consistency is symbolic grounding in the causal generator of the world's dynamics.
  arXiv  Detail & Related papers  (2026-06-09T23:00:48Z)
• Consistent Geometric Deep Learning via Hilbert Bundles and Cellular Sheaves [68.683840670745]
  We introduce a novel convolutional learning framework for possibly infinite-dimensional signals supported on a manifold.<n>We make HilbNets and, more generally, the convolution operation, implementable via a two-stage sampling procedure.<n>Our results broaden the scope of geometric learning as a whole by lifting classical Laplacian-based frameworks to settings where the signal at each point lives in its own Hilbert space.
  arXiv  Detail & Related papers  (2026-05-07T15:08:58Z)
• Reasoning on the Manifold: Bidirectional Consistency for Self-Verification in Diffusion Language Models [30.607422839762553]
  Bidirectional Manifold Consistency (BMC) is a training-free, unsupervised metric.<n>BMC serves as a robust discriminator of solution validity without ground truth answer.<n>Our results establish intrinsic geometric stability as a robust indicator of correctness for dLLMs.
  arXiv  Detail & Related papers  (2026-04-17T10:17:16Z)
• Score-Guided Proximal Projection: A Unified Geometric Framework for Rectified Flow Editing [1.0312968200748118]
  Rectified Flow models achieve state-of-the-art generation quality, yet controlling them for precise tasks remains a challenge.<n>Current approaches bifurcate into inversion-based guidance, which suffers from "geometric locking"<n>We propose Score-Guided Proximal Projection, a unified framework that bridges the gap between deterministic optimization and contraction sampling.
  arXiv  Detail & Related papers  (2026-03-05T23:44:45Z)
• The Procrustean Bed of Time Series: The Optimization Bias of Point-wise Loss [53.542743390809356]
  This paper aims to provide a first-principles analysis of the Expectation of Optimization Bias (EOB)<n>Our analysis reveals a fundamental paradigm paradox: the more deterministic and structured the time series, the more severe the bias by point-wise loss function.<n>We present a concrete solution that simultaneously achieves both principles via DFT or DWT.
  arXiv  Detail & Related papers  (2025-12-21T06:08:22Z)
• Latent Iterative Refinement Flow: A Geometric-Constrained Approach for Few-Shot Generation [5.062604189239418]
  We introduce Latent Iterative Refinement Flow (LIRF), a novel approach to few-shot generation.<n>LIRF establishes a stable latent space using an autoencoder trained with our novel textbfmanifold-preservation loss.<n>Within this cycle, candidate samples are refined by a geometric textbfcorrection operator, a provably contractive mapping.
  arXiv  Detail & Related papers  (2025-09-24T08:57:21Z)
• Generalized Linear Bandits: Almost Optimal Regret with One-Pass Update [70.38810219913593]
  We study the generalized linear bandit (GLB) problem, a contextual multi-armed bandit framework that extends the classical linear model by incorporating a non-linear link function.<n>GLBs are widely applicable to real-world scenarios, but their non-linear nature introduces significant challenges in achieving both computational and statistical efficiency.<n>We propose a jointly efficient algorithm that attains a nearly optimal regret bound with $mathcalO(1)$ time and space complexities per round.
  arXiv  Detail & Related papers  (2025-07-16T02:24:21Z)
• Generalized Linear Markov Decision Process [9.219628236765933]
  Generalized Linear MDP (GLMDP) framework models rewards using generalized linear models (GLMs)<n>We develop two offline RL algorithms: Generalized Pessimistic Value Iteration (GPEVI) and a semi-supervised variant (SS-GPEVI)<n>Our algorithms achieve theoretical guarantees on policy suboptimality and demonstrate improved sample efficiency in settings where reward labels are expensive or limited.
  arXiv  Detail & Related papers  (2025-06-01T03:50:41Z)
• Disentangled Representation Learning with the Gromov-Monge Gap [65.73194652234848]
  Learning disentangled representations from unlabelled data is a fundamental challenge in machine learning.<n>We introduce a novel approach to disentangled representation learning based on quadratic optimal transport.<n>We demonstrate the effectiveness of our approach for quantifying disentanglement across four standard benchmarks.
  arXiv  Detail & Related papers  (2024-07-10T16:51:32Z)
• Generative Locally Linear Embedding [5.967999555890417]
  Linear Locally Embedding (LLE) is a nonlinear spectral dimensionality reduction and manifold learning method. We propose two novel generative versions of LLE, named Generative LLE (GLLE) Our simulations show that the proposed GLLE methods work effectively in unfolding and generating submanifolds of data.
  arXiv  Detail & Related papers  (2021-04-04T02:59:39Z)
• On the Stability of Nonlinear Receding Horizon Control: A Geometric Perspective [72.7951562665449]
  widespread adoption of nonlinear Receding Control (RHC) strategies by industry has to more than 30 years. This paper takes the first step towards understanding the role of global geometry in the role of global-based control.
  arXiv  Detail & Related papers  (2021-03-27T22:59:37Z)
• GELATO: Geometrically Enriched Latent Model for Offline Reinforcement Learning [54.291331971813364]
  offline reinforcement learning approaches can be divided into proximal and uncertainty-aware methods. In this work, we demonstrate the benefit of combining the two in a latent variational model. Our proposed metrics measure both the quality of out of distribution samples as well as the discrepancy of examples in the data.
  arXiv  Detail & Related papers  (2021-02-22T19:42:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.