LpWM: A Case for Sparse Representations in World Models

• URL: http://arxiv.org/abs/2608.22764v1
• Date: Mon, 24 Aug 2026 03:37:39 GMT
• Title: LpWM: A Case for Sparse Representations in World Models
• Abstract summary: We show that sparse representations can make action-conditioned latent dynamics easier to model.<n>We introduce LpModelWorld (LpWM), a JEPA model regularized with Rectified Distribution Matching Regularization (RDMReg)<n>On PushT, sparse LpWM outperforms dense LeWM by up to 57% in planning success at intermediate predictor capacities.
• Score: 45.053365014723404
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Joint-embedding predictive architectures (JEPAs) learn latent dynamics for planning and avoid representation collapse by matching features to maximum-entropy distributions such as isotropic Gaussians, yielding dense representations. However, it is unclear whether dense representations are the most favorable geometry for modeling dynamics. In this work, we ask whether a different geometry, sparse representations, can make action-conditioned latent dynamics easier to model, and what dynamical structure emerges from such representations. We first show that nonlinear Lipschitz dynamics can be approximated arbitrarily well by action-conditioned linear dynamics in a sufficiently high-dimensional one-hot latent space, with rollout error vanishing as the dimension grows. This motivates distributed sparse representations as a practical relaxation of one-hot sparsity. We introduce LpWorldModel (LpWM), a JEPA model regularized with Rectified Distribution Matching Regularization (RDMReg) to match encoder features to a Rectified Generalized Gaussian distribution, yielding non-negative sparse codes. Empirically, sparsity lowers the predictor complexity required for successful planning: on PushT, sparse LpWM outperforms dense LeWM by up to 57% in planning success at intermediate predictor capacities. This advantage also extends beyond Gaussian distribution matching, with LpWM outperforming dense VICReg representations across multiple predictor families. We further find that the learned sparse representations are mode-factored, with support encoding discrete dynamical regimes and feature magnitudes capturing continuous within-regime state. Together, these results suggest that sparse representations can reduce the predictor complexity required for control while revealing interpretable structure.

Related papers

• The Hyperspherical Geometry of CLIP Latent Space: A Semantic Mixture Model [73.29820477295043]
  Contrastive Language-Image Pretraining (CLIP) representations form a semantic embedding space governed by cosine similarity.<n>We propose a principled density model for the CLIP latent space based on Mixtures of von Mises-Fisher (MovMF) distributions.
  arXiv  Detail & Related papers  (2026-07-15T10:05:53Z)
• ThinkJEPA: Empowering Latent World Models with Large Vision-Language Reasoning Model [53.15040805435013]
  Vision-language models (VLMs) provide strong semantic grounding and general knowledge by reasoning over uniformly sampled frames.<n>We propose a VLM-guided JEPA-style latent world modeling framework that combines dense-frame dynamics modeling with long-horizon semantic guidance.
  arXiv  Detail & Related papers  (2026-03-23T17:59:42Z)
• Rectified LpJEPA: Joint-Embedding Predictive Architectures with Sparse and Maximum-Entropy Representations [53.61624356747686]
  Joint-Embedding Predictive Architectures (JEPA) learn view-invariant representations and admit projection-based distribution matching for collapse prevention.<n>Existing approaches regularize representations towards isotropic Gaussian distributions, but inherently favor dense representations and fail to capture the key property of sparsity observed in efficient representations.<n>We introduce Rectified Distribution Matching Regularization (RDMReg), a sliced two-sample distribution-matching loss that aligns representations to a Rectified Generalized Gaussian (RGG) distribution.
  arXiv  Detail & Related papers  (2026-02-01T21:49:30Z)
• Predicting Forced Responses of Probability Distributions via the Fluctuation-Dissipation Theorem and Generative Modeling [0.0]
  We present a data-driven framework for estimating the response of higher-order moments of nonlinear systems to small external perturbations.<n>We combine GFDT with score-based generative modeling to estimate the system's score function directly from data.<n>Our method is validated on several models relevant to climate dynamics.
  arXiv  Detail & Related papers  (2025-04-17T20:54:33Z)
• Expectation propagation for the smoothing distribution in dynamic probit [1.6114012813668932]
  The smoothing distribution of dynamic probit models with Gaussian state dynamics was recently proved to belong to the unified skew-normal family. We derive an efficient EP routine to perform inference for such a distribution. We show that the proposed approximation leads to accuracy gains over available approximate algorithms in a financial illustration.
  arXiv  Detail & Related papers  (2023-09-04T14:49:45Z)
• Sparse Model Soups: A Recipe for Improved Pruning via Model Averaging [24.64264715041198]
  We introduce Sparse Model Soups (SMS), a novel method for merging sparse models by initiating each prune-retrain cycle with the averaged model from the previous phase. SMS preserves sparsity, exploits sparse network benefits, is modular and fully parallelizable, and substantially improves IMP's performance.
  arXiv  Detail & Related papers  (2023-06-29T08:49:41Z)
• Hierarchical Integration Diffusion Model for Realistic Image Deblurring [71.76410266003917]
  Diffusion models (DMs) have been introduced in image deblurring and exhibited promising performance. We propose the Hierarchical Integration Diffusion Model (HI-Diff), for realistic image deblurring. Experiments on synthetic and real-world blur datasets demonstrate that our HI-Diff outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-05-22T12:18:20Z)
• PAC Reinforcement Learning for Predictive State Representations [60.00237613646686]
  We study online Reinforcement Learning (RL) in partially observable dynamical systems. We focus on the Predictive State Representations (PSRs) model, which is an expressive model that captures other well-known models. We develop a novel model-based algorithm for PSRs that can learn a near optimal policy in sample complexity scalingly.
  arXiv  Detail & Related papers  (2022-07-12T17:57:17Z)
• Generative Model without Prior Distribution Matching [26.91643368299913]
  Variational Autoencoder (VAE) and its variations are classic generative models by learning a low-dimensional latent representation to satisfy some prior distribution. We propose to let the prior match the embedding distribution rather than imposing the latent variables to fit the prior.
  arXiv  Detail & Related papers  (2020-09-23T09:33:24Z)
• Stochastic spectral embedding [0.0]
  We propose a novel sequential adaptive surrogate modeling method based on "stochastic spectral embedding" (SSE) We show how the method compares favorably against state-of-the-art sparse chaos expansions on a set of models with different complexity and input dimension.
  arXiv  Detail & Related papers  (2020-04-09T11:00:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.