Traj-LeWM: Path-Aware World-Model Planning via Latent Trajectory Cost

• URL: http://arxiv.org/abs/2608.14125v1
• Date: Fri, 14 Aug 2026 09:32:38 GMT
• Title: Traj-LeWM: Path-Aware World-Model Planning via Latent Trajectory Cost
• Abstract summary: We propose Traj-LeWM, which retains LeWM's objective and endpoint score while introducing a goal-conditioned latent trajectory cost.<n>During training, LTC-based trajectory-preference supervision complements next-step prediction in shaping the shared representation.<n>With joint endpoint-plus-LTC scoring, Traj-LeWM outperforms LeWM on Push-T, OGBench-Cube, Reacher, and Two-Room.
• Score: 23.64925215430669
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: LeWM is a lightweight visual world model that learns latent dynamics end-to-end from pixels and ranks candidate action sequences by the distance between their predicted endpoints and the goal. However, LeWM has two limitations. First, during training, it learns local next-step transitions without evaluating complete trajectories relative to the task goal. Second, during planning, it ranks candidates solely by predicted endpoint distance. Because model predictions may differ from actual execution outcomes, the candidate whose predicted endpoint is closest to the goal may not perform best when executed in the environment. The evolution of the complete predicted trajectory can therefore provide complementary information beyond endpoint distance. To address these limitations, we propose Traj-LeWM, which retains LeWM's local-dynamics objective and endpoint score while introducing a goal-conditioned latent trajectory cost (LTC) that aggregates trajectory-level information as a complementary signal. During training, LTC-based trajectory-preference supervision complements next-step prediction in shaping the shared representation. During planning, LTC is combined with endpoint distance to incorporate intermediate-path information into candidate ranking. With joint endpoint-plus-LTC scoring, Traj-LeWM outperforms LeWM on Push-T, OGBench-Cube, Reacher, and Two-Room by $3$, $14$, $7$, and $7$ percentage points, respectively. Controlled experiments and ablations further verify the complementary roles of trajectory-level representation shaping and path-aware candidate ranking.

Related papers

• SCALE: State-Calibrated Latent Embeddings for JEPA Planning in the Right Geometry [14.742804674497128]
  Two strategies for obtaining non-collapsed representations are to inherit a pretrained feature space, and to learn an embedding end to end with anti-collapse regularization.<n>We show that planning depends not only on whether task-relevant information is present, but also on whether it shapes the geometry consumed by the planner.
  arXiv  Detail & Related papers  (2026-08-17T08:58:55Z)
• Temporal-Distance JEPA: Plan-Aware Representation Learning for Latent World Model Predictive Control [8.39933163242427]
  JEPAs learn world models by predicting in representation space rather than reconstructing pixels.<n>Prior JEPA planners often inherit ranking from embedding geometry, typically latent Euclidean distance.<n>We propose Temporal-Distance-JEPA, which retains the LeWM encoder-predictor backbone and mines a directed temporal cost from reward-free trajectories.
  arXiv  Detail & Related papers  (2026-07-28T06:38:16Z)
• Beyond Waypoints: A Trajectory-Centric Waypointing Paradigm for Vision-Language Navigation [79.12557132788139]
  Vision-Language Navigation in Continuous Environments (VLN-CE) requires agents to follow natural-language instructions while navigating in real-world-like environments.<n>Most VLN-CE approach-es adopt a three-stage framework: a waypoint predictor proposes waypoints, and a navigator selects the best waypoint.<n>We introduce a novel paradigm called Trajectory Waypoint, which grounds each candidate waypoint in an executable trajectory.
  arXiv  Detail & Related papers  (2026-06-05T13:11:39Z)
• CoWTracker: Tracking by Warping instead of Correlation [53.834673070954494]
  We propose a dense point tracker that eschews cost volumes in favor of warping.<n>Inspired by recent advances in optical flow, our approach iteratively refines track estimates by warping features from the target frame to the query frame based on the current estimate.<n>Our model is simple and achieves state-of-the-art performance on standard dense point tracking benchmarks, including TAP-Vid-DAVIS, TAP-Vid-Kinetics, and Robo-TAP.
  arXiv  Detail & Related papers  (2026-02-04T18:58:59Z)
• GUIDE-CoT: Goal-driven and User-Informed Dynamic Estimation for Pedestrian Trajectory using Chain-of-Thought [9.572859785331307]
  We propose Goal-driven and User-Informed Dynamic Estimation for pedestrian trajectory using Chain-of-Thought (GUIDE-CoT)<n>Our approach integrates two innovative modules: (1) a goal-oriented visual prompt, which enhances goal prediction accuracy combining visual prompts with a pretrained visual encoder, and (2) a chain-of-thought (CoT) LLM for trajectory generation, which generates realistic trajectories toward the predicted goal.<n>Our method achieves state-of-the-art performance, delivering both high accuracy and greater adaptability in pedestrian trajectory prediction.
  arXiv  Detail & Related papers  (2025-03-10T01:39:24Z)
• PRET: Planning with Directed Fidelity Trajectory for Vision and Language Navigation [30.710806048991923]
  Vision and language navigation is a task that requires an agent to navigate according to a natural language instruction. Recent methods predict sub-goals on constructed topology map at each step to enable long-term action planning. We propose an alternative method that facilitates navigation planning by considering the alignment between instructions and directed fidelity trajectories.
  arXiv  Detail & Related papers  (2024-07-16T08:22:18Z)
• Affordances-Oriented Planning using Foundation Models for Continuous Vision-Language Navigation [64.84996994779443]
  We propose a novel Affordances-Oriented Planner for continuous vision-language navigation (VLN) task. Our AO-Planner integrates various foundation models to achieve affordances-oriented low-level motion planning and high-level decision-making. Experiments on the challenging R2R-CE and RxR-CE datasets show that AO-Planner achieves state-of-the-art zero-shot performance.
  arXiv  Detail & Related papers  (2024-07-08T12:52:46Z)
• Transforming Model Prediction for Tracking [109.08417327309937]
  Transformers capture global relations with little inductive bias, allowing it to learn the prediction of more powerful target models. We train the proposed tracker end-to-end and validate its performance by conducting comprehensive experiments on multiple tracking datasets. Our tracker sets a new state of the art on three benchmarks, achieving an AUC of 68.5% on the challenging LaSOT dataset.
  arXiv  Detail & Related papers  (2022-03-21T17:59:40Z)
• Goal-GAN: Multimodal Trajectory Prediction Based on Goal Position Estimation [1.20855096102517]
  We present Goal-GAN, an interpretable and end-to-end trainable model for human trajectory prediction. Inspired by human navigation, we model the task of trajectory prediction as an intuitive two-stage process.
  arXiv  Detail & Related papers  (2020-10-02T17:17:45Z)
• TNT: Target-driveN Trajectory Prediction [76.21200047185494]
  We develop a target-driven trajectory prediction framework for moving agents. We benchmark it on trajectory prediction of vehicles and pedestrians. We outperform state-of-the-art on Argoverse Forecasting, INTERACTION, Stanford Drone and an in-house Pedestrian-at-Intersection dataset.
  arXiv  Detail & Related papers  (2020-08-19T06:52:46Z)
• Long-Horizon Visual Planning with Goal-Conditioned Hierarchical Predictors [124.30562402952319]
  The ability to predict and plan into the future is fundamental for agents acting in the world. Current learning approaches for visual prediction and planning fail on long-horizon tasks. We propose a framework for visual prediction and planning that is able to overcome both of these limitations.
  arXiv  Detail & Related papers  (2020-06-23T17:58:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.