SCALE: State-Calibrated Latent Embeddings for JEPA Planning in the Right Geometry

• URL: http://arxiv.org/abs/2608.16287v2
• Date: Wed, 26 Aug 2026 17:10:59 GMT
• Title: SCALE: State-Calibrated Latent Embeddings for JEPA Planning in the Right Geometry
• Abstract summary: Two strategies for obtaining non-collapsed representations are to inherit a pretrained feature space, and to learn an embedding end to end with anti-collapse regularization.<n>We show that planning depends not only on whether task-relevant information is present, but also on whether it shapes the geometry consumed by the planner.
• Score: 14.742804674497128
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Joint-embedding predictive world models plan by scoring predicted terminal embeddings against a goal embedding using a cost defined on the representation itself. Two prominent strategies for obtaining non-collapsed representations are to inherit a pretrained feature space, as in DINO-WM, and to learn an embedding end to end with anti-collapse regularization, as in LeWorldModel (LeWM) with SIGReg. These strategies show complementary strengths across tasks. Although task-relevant state is decodable from the full embeddings of both models, DINO-WM's leading principal components usually retain substantially more state information than LeWM's. Because Euclidean planning costs are dominated by high-variance directions, this difference affects how strongly state can influence candidate selection. We propose SCALE (State-CAlibrated Latent Embeddings) to give the end-to-end LeWM representation the favorable geometric property observed in DINO-WM. SCALE induces this property by correlating sampled pairwise latent distances with distances in a standardized task-relevant state space, without replacing LeWM's learned encoder. Across five tasks, three planning solvers, and five compute budgets, SCALE improves every task--solver average over LeWM. A latent-to-state regression control matches or exceeds SCALE's full-embedding decodability yet leaves latent--state distance alignment essentially unchanged and yields less consistent planning gains. SCALE adds a single lightweight training-time regularizer and no planning-time overhead. These results show that planning depends not only on whether task-relevant information is present, but also on whether it shapes the geometry consumed by the planner.

Related papers

• Orthogonal JEPA: Factorized Predictive States for Latent World Models [95.20158869896393]
  We introduce method, a latent world-modeling framework based on predictive factorization.<n>We show that method can allocate redundant capacity to dominant signals while providing weak or conflicting gradients to less dominant predictive structure.<n>Experiments on controlled vision, single-cell transcriptomics, longitudinal health records, continuous control, and molecular dynamics evaluate representation quality, forecasting, planning, and long-horizon stability.
  arXiv  Detail & Related papers  (2026-08-20T13:59:57Z)
• Decision-Metric Alignment in Latent World Models: Diagnostics and Action-Conditioned Objectives for MPC Planning [5.704305798718388]
  JEPA-style latent world models can use Euclidean distance to a goal latent as the cost for model-predictive control (MPC)<n>We introduce Plan-Real Spearman, which measures latent--real rank agreement on random plans, and CEM-stage Spearman, which measures the same agreement as cross-entropy-method (CEM) search concentrates its proposal.
  arXiv  Detail & Related papers  (2026-08-19T09:56:39Z)
• VIScore: Diagnosing Planning-Relevant Quality in Latent World Models [21.894781659564817]
  Regulating the latent space to an isotropic Gaussian distribution provides a stable and information-maximized landscape for world model planning.<n>We study this by comparing SIGReg and VISReg, two regularization loss functions with the same distribution target but different properties.<n>Compared with SIGReg, VISReg has more flexibility in controlling the weights of center, scale, and shape regularization, and a larger batch size brings a finer distribution approximation.<n>We propose the Veracity-Influence-Sobriety score (VIScore), a metric that quantifies the reachability and capacity of a predictor given the encoded
  arXiv  Detail & Related papers  (2026-08-11T17:34:10Z)
• Temporal-Distance JEPA: Plan-Aware Representation Learning for Latent World Model Predictive Control [8.39933163242427]
  JEPAs learn world models by predicting in representation space rather than reconstructing pixels.<n>Prior JEPA planners often inherit ranking from embedding geometry, typically latent Euclidean distance.<n>We propose Temporal-Distance-JEPA, which retains the LeWM encoder-predictor backbone and mines a directed temporal cost from reward-free trajectories.
  arXiv  Detail & Related papers  (2026-07-28T06:38:16Z)
• Fast LeWorldModel [5.938994204098151]
  We propose Fast LeWorldModel (Fast-LeWM), a fast latent world model that replaces repeated local rollout with action prediction.<n>By making action prefixes the basic prediction unit, Fast-LeWM directly models action effects accumulated to different extents over multiple horizons.<n>Fast-LeWM improves average success over LeWM while substantially reducing planning time, achieving lower open-loop latent loss.
  arXiv  Detail & Related papers  (2026-06-24T17:59:59Z)
• Sensorimotor World Models: Perception for Action via Inverse Dynamics [59.121736411156384]
  We introduce a sensorimotor world model trained end-to-end with inverse dynamics regularization.<n>It prevents representation collapse and induces action-aligned representations.<n>It learns compact, interpretable latent spaces and enables competitive planning performance across simple 2D and 3D control tasks.
  arXiv  Detail & Related papers  (2026-06-18T11:25:16Z)
• Latent Geometry Beyond Search: Amortizing Planning in World Models [17.571800971505787]
  We show that a learned representation can simplify control, rather than merely enabling prediction.<n>We replace iterative planning with a lightweight Goal-Conditioned Inverse Dynamics Model (GC-IDM)<n>Our controller matches or exceeds CEM in seven of eight environment-protocol settings while reducing per-decision cost by 100-130x.
  arXiv  Detail & Related papers  (2026-05-09T06:36:23Z)
• LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels [49.35636088613484]
  Joint Embedding Predictive Architectures (JEPAs) offer a compelling framework for learning world models in compact latent spaces.<n>We introduce LeModelWorld, the first JEPA that trains stably end-to-end from raw pixels using only two loss terms.<n>With 15M parameters trainable on a single GPU in a few hours, LeWM plans up to 48x faster than foundation-model-based world models.
  arXiv  Detail & Related papers  (2026-03-13T19:48:14Z)
• ProAct: Agentic Lookahead in Interactive Environments [56.50613398808361]
  ProAct is a framework that enables agents to internalize accurate lookahead reasoning through a two-stage training paradigm.<n>We introduce Grounded LookAhead Distillation (GLAD), where the agent undergoes supervised fine-tuning on trajectories derived from environment-based search.<n>We also propose the Monte-Carlo Critic (MC-Critic), a plug-and-play auxiliary value estimator designed to enhance policy-gradient algorithms.
  arXiv  Detail & Related papers  (2026-02-05T05:45:16Z)
• Agentic World Modeling for 6G: Near-Real-Time Generative State-Space Reasoning [70.56067503630486]
  We argue that sixth-generation (6G) intelligence is not fluent token prediction but calibrated the capacity to imagine and choose.<n>We show that WM-MS3M cuts mean absolute error (MAE) by 1.69% versus MS3M with 32% fewer parameters and similar latency, and achieves 35-80% lower root mean squared error (RMSE) than attention/hybrid baselines with 2.3-4.1x faster inference.
  arXiv  Detail & Related papers  (2025-11-04T17:22:22Z)
• Unified World Models: Memory-Augmented Planning and Foresight for Visual Navigation [69.94565127141483]
  Current approaches separate navigation planning from visual world modeling, leading to state-action misalignment and limited adaptability.<n>We propose UniWM, a unified, memory-augmented world model integrating egocentric visual foresight and planning within a single multimodal autoregressive backbone.<n>We show that UniWM substantially improves navigation success rates by up to 30%, significantly reduces trajectory errors compared to strong baselines, and exhibits impressive zero-shot generalization on the unseen TartanDrive dataset.
  arXiv  Detail & Related papers  (2025-10-09T18:18:11Z)
• LaDi-WM: A Latent Diffusion-based World Model for Predictive Manipulation [45.02469804709771]
  We propose LaDi-WM, a world model that predicts the latent space of future states using diffusion modeling.<n>We show that LaDi-WM significantly enhances policy performance by 27.9% on the LIBERO-LONG benchmark and 20% on the real-world scenario.
  arXiv  Detail & Related papers  (2025-05-13T04:42:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.