FF-JEPA: Long-Horizon Planning in World Models with Latent Planners

• URL: http://arxiv.org/abs/2606.09311v1
• Date: Mon, 08 Jun 2026 10:16:59 GMT
• Title: FF-JEPA: Long-Horizon Planning in World Models with Latent Planners
• Abstract summary: Joint Embedding Predictive Architectures (JEPAs) have shown promising world modeling capabilities.<n>We propose Forward-Forward-JEPA (FF-JEPA), a hierarchical approach leveraging two forward dynamics models.<n>Preliminary results on PushT demonstrate that FF-JEPA successfully overcomes flat world models' long-horizon collapse.
• Score: 30.722155570120766
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Joint Embedding Predictive Architectures (JEPAs) have shown promising world modeling capabilities, enabling planning in latent space by optimizing action trajectories using methods like the Cross-Entropy Method (CEM). These methods are, however, too computationally expensive and ineffective for long-horizon planning. Furthermore, these methods typically require an explicit image of the goal state, which is not always possible in real-world tasks. In this work, we tackle these limitations by proposing Forward-Forward-JEPA (FF-JEPA), a hierarchical approach leveraging two forward dynamics models. Alongside a standard action-conditioned forward model, we introduce an action-free latent planner that predicts the next subgoal given the current state. This approach removes the need for goal images and enables long-horizon planning by decomposing complex trajectories into a sequence of tractable, short-term optimization problems. Preliminary results on PushT demonstrate that FF-JEPA successfully overcomes flat world models' long-horizon collapse, highlighting this approach as a promising direction for goal-free planning.

Related papers

• AdaJEPA: An Adaptive Latent World Model [42.0759614364807]
  We propose AdaJEPA, an adaptive latent world model that performs test-time adaptation within the closed loop of model predictive control (MPC)<n>After training, AdaJEPA plans and executes the first action chunk, uses the observed next-state transition as a self-supervised adaptation signal, and replans with the updated model.<n>This closed-loop update continuously recalibrates the world model without additional expert demonstrations.
  arXiv  Detail & Related papers  (2026-06-30T17:53:48Z)
• Beyond the Next Step: Variable-Length Latent World Models for Long-Horizon Planning [40.09669638091686]
  We propose a framework that learns to predict future latent states conditioned on action sequences of variable lengths.<n>Experiments show that Variable-length Latent World Models significantly improve latent space world models.
  arXiv  Detail & Related papers  (2026-06-19T21:50:15Z)
• HAD: Combining Hierarchical Diffusion with Metric-Decoupled RL for End-to-End Driving [51.268878540511054]
  We propose HAD, an end-to-end planning framework with a Hierarchical Diffusion Policy.<n>We show that HAD achieves new state-of-the-art performance on both NAVSIM and HUGSIM.
  arXiv  Detail & Related papers  (2026-04-04T04:12:47Z)
• Hierarchical Planning with Latent World Models [49.82095442483551]
  hierarchical planning achieves higher success while requiring up to 4x less planning-time compute.<n>We demonstrate that this hierarchical approach enables zero-shot control on real-world non-greedy robotic tasks.
  arXiv  Detail & Related papers  (2026-04-03T17:32:36Z)
• Value-guided action planning with JEPA world models [44.84158001773079]
  Building deep learning models that can reason about their environment requires capturing its underlying dynamics.<n>Joint-Embedded Predictive Architectures (JEPA) provide a promising framework to model such dynamics.<n>We propose an approach to enhance planning with JEPA world models by shaping their representation space.
  arXiv  Detail & Related papers  (2025-12-28T20:17:49Z)
• Closing the Train-Test Gap in World Models for Gradient-Based Planning [64.36544881136405]
  We propose improved methods for training world models that enable efficient gradient-based planning.<n>At test time, our approach outperforms or matches the classical gradient-free cross-entropy method.
  arXiv  Detail & Related papers  (2025-12-10T18:59:45Z)
• Deep Active Inference Agents for Delayed and Long-Horizon Environments [1.693200946453174]
  AIF agents rely on accurate immediate predictions and exhaustive planning, a limitation that is exacerbated in delayed environments.<n>We propose a generative-policy architecture featuring a multi-step latent transition that lets the generative model predict an entire horizon in a single look-ahead.<n>We evaluate our agent in an environment that mimics a realistic industrial scenario with delayed and long-horizon settings.
  arXiv  Detail & Related papers  (2025-05-26T11:50:22Z)
• Extendable Long-Horizon Planning via Hierarchical Multiscale Diffusion [62.91968752955649]
  This paper tackles a novel problem, extendable long-horizon planning-enabling agents to plan trajectories longer than those in training data without compounding errors.<n>We propose an augmentation method that iteratively generates longer trajectories by stitching shorter ones.<n>HM-Diffuser trains on these extended trajectories using a hierarchical structure, efficiently handling tasks across multiple temporal scales.
  arXiv  Detail & Related papers  (2025-03-25T22:52:46Z)
• Plan-and-Act: Improving Planning of Agents for Long-Horizon Tasks [36.63527489464188]
  Plan-and-Act is a framework that incorporates explicit planning into large language models (LLMs)<n>Plan-and-Act consists of a Planner model which generates structured, high-level plans to achieve user goals, and an Executor model that translates these plans into environment-specific actions.<n>We present a state-of-the-art 57.58% success rate on the WebArena-Lite benchmark as well as a text-only state-of-the-art 81.36% success rate on WebVoyager.
  arXiv  Detail & Related papers  (2025-03-12T17:40:52Z)
• Model-Based Reinforcement Learning via Latent-Space Collocation [110.04005442935828]
  We argue that it is easier to solve long-horizon tasks by planning sequences of states rather than just actions. We adapt the idea of collocation, which has shown good results on long-horizon tasks in optimal control literature, to the image-based setting by utilizing learned latent state space models.
  arXiv  Detail & Related papers  (2021-06-24T17:59:18Z)
• Long-Horizon Visual Planning with Goal-Conditioned Hierarchical Predictors [124.30562402952319]
  The ability to predict and plan into the future is fundamental for agents acting in the world. Current learning approaches for visual prediction and planning fail on long-horizon tasks. We propose a framework for visual prediction and planning that is able to overcome both of these limitations.
  arXiv  Detail & Related papers  (2020-06-23T17:58:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.