Latent Energy Action Planning with World Models

• URL: http://arxiv.org/abs/2609.03294v1
• Date: Thu, 03 Sep 2026 02:41:36 GMT
• Title: Latent Energy Action Planning with World Models
• Abstract summary: We introduce Latent Energy Action Planning (LEAP), which treats the complete action horizon as a differentiable variable.<n>LEAP couples terminal latent goal matching with a terminal-window state energy.<n>A quasi-Newton solver refines actions through the autoregressive rollout, and post-optimization projection enforces the admissible action range.
• Score: 16.141037661392115
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Latent world models support efficient model predictive control from high-dimensional observations, yet optimizing a single learned latent objective can favor action sequences whose decoder-predicted terminal descriptor does not match the goal descriptor. We introduce Latent Energy Action Planning (LEAP), which treats the complete action horizon as a differentiable variable and optimizes it through a frozen LeWorldModel (LeWM). LEAP couples terminal latent goal matching with a terminal-window state energy. Low energy requires the predicted terminal latent to agree with the goal latent and the decoder-predicted terminal descriptor to agree with the goal descriptor. A frozen goal-conditioned proposal initializes the search, a quasi-Newton solver refines actions through the autoregressive rollout, and post-optimization projection enforces the admissible action range. Across four control domains using the officially released LeWM checkpoints, the complete LEAP planning system raises mean success from 77.5% for LeWM planned with the cross-entropy method (LeWM+CEM) to 94.8% under a matched protocol, a 17.3-percentage-point improvement, while retaining the frozen LeWM representation and predictor.

Related papers

• Decision-Metric Alignment in Latent World Models: Diagnostics and Action-Conditioned Objectives for MPC Planning [5.704305798718388]
  JEPA-style latent world models can use Euclidean distance to a goal latent as the cost for model-predictive control (MPC)<n>We introduce Plan-Real Spearman, which measures latent--real rank agreement on random plans, and CEM-stage Spearman, which measures the same agreement as cross-entropy-method (CEM) search concentrates its proposal.
  arXiv  Detail & Related papers  (2026-08-19T09:56:39Z)
• Traj-LeWM: Path-Aware World-Model Planning via Latent Trajectory Cost [23.64925215430669]
  We propose Traj-LeWM, which retains LeWM's objective and endpoint score while introducing a goal-conditioned latent trajectory cost.<n>During training, LTC-based trajectory-preference supervision complements next-step prediction in shaping the shared representation.<n>With joint endpoint-plus-LTC scoring, Traj-LeWM outperforms LeWM on Push-T, OGBench-Cube, Reacher, and Two-Room.
  arXiv  Detail & Related papers  (2026-08-14T09:32:38Z)
• QQWorld: Quantile-Quantile Matching for World Model Regularization [7.147705178168053]
  LeWorldModel (LeWM) regularizes its latents toward an isotropic Gaussian using the Epps-Pulley (EP) objective.<n>We show that the corrective gradients of EP rapidly vanish for isolated tail samples, leaving heavy-tailed deviations insufficiently controlled.<n>We propose QQWorld, which replaces EP with a quantile-quantile matching objective that directly aligns projected latent samples with rank-matched Gaussian quantiles.
  arXiv  Detail & Related papers  (2026-07-30T16:00:39Z)
• Fast LeWorldModel [5.938994204098151]
  We propose Fast LeWorldModel (Fast-LeWM), a fast latent world model that replaces repeated local rollout with action prediction.<n>By making action prefixes the basic prediction unit, Fast-LeWM directly models action effects accumulated to different extents over multiple horizons.<n>Fast-LeWM improves average success over LeWM while substantially reducing planning time, achieving lower open-loop latent loss.
  arXiv  Detail & Related papers  (2026-06-24T17:59:59Z)
• AnchorRefine: Synergy-Manipulation Based on Trajectory Anchor and Residual Refinement for Vision-Language-Action Models [60.04879435087352]
  Vision-language-action (VLA) policies generate actions within a single unified space.<n>We propose AnchorRefine, a hierarchical framework that factorizes VLA action modeling into trajectory anchor and residual refinement.<n> Experiments on LIBERO, CALVIN, and real-robot tasks demonstrate that AnchorRefine consistently improves both regression-based and diffusion-based VLA backbones.
  arXiv  Detail & Related papers  (2026-04-20T04:25:24Z)
• Grounded World Model for Semantically Generalizable Planning [94.53923128709965]
  We learn a Grounded World Model (GWM) in a vision-language-aligned latent space.<n>Each proposed action is scored based on how close its future outcome is to the task instruction.<n>On the proposed WISER benchmark, GWM-MPC achieves a 87% success rate on the test set.
  arXiv  Detail & Related papers  (2026-04-13T17:25:41Z)
• World Action Verifier: Self-Improving World Models via Forward-Inverse Asymmetry [82.93104394404781]
  General-purpose world models promise scalable policy evaluation, optimization, and planning.<n>We propose World Action Verifier (WAV), a framework that enables world models to identify their own prediction errors and self-improve.
  arXiv  Detail & Related papers  (2026-04-02T12:48:36Z)
• Matching Features, Not Tokens: Energy-Based Fine-Tuning of Language Models [102.20309135516186]
  Cross-entropy (CE) training provides dense and scalable supervision for language models.<n>We introduce a feature-matching objective for language-model fine-tuning.<n>To optimize this objective efficiently, we propose energy-based fine-tuning.
  arXiv  Detail & Related papers  (2026-03-12T17:57:50Z)
• Learning to Reason and Navigate: Parameter Efficient Action Planning with Large Language Models [63.765846080050906]
  This paper proposes a novel parameter-efficient action planner using large language models (PEAP-LLM) to generate a single-step instruction at each location.<n>Experiments show the superiority of our proposed model on REVERIE compared to the previous state-of-the-art.
  arXiv  Detail & Related papers  (2025-05-12T12:38:20Z)
• Modular Deep Reinforcement Learning for Continuous Motion Planning with Temporal Logic [59.94347858883343]
  This paper investigates the motion planning of autonomous dynamical systems modeled by Markov decision processes (MDP) The novelty is to design an embedded product MDP (EP-MDP) between the LDGBA and the MDP. The proposed LDGBA-based reward shaping and discounting schemes for the model-free reinforcement learning (RL) only depend on the EP-MDP states.
  arXiv  Detail & Related papers  (2021-02-24T01:11:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.