LeFlow: Generative Latent Flow Planning for World Models

• URL: http://arxiv.org/abs/2608.24855v1
• Date: Tue, 25 Aug 2026 17:42:30 GMT
• Title: LeFlow: Generative Latent Flow Planning for World Models
• Abstract summary: We present LeFlow, which learns a reusable latent trajectory prior operating directly in the latent dynamics space from the world model.<n>Across four major goal-conditioned pixel-control benchmarks, LeFlow replaces iterative action-space optimization with amortized latent planning.<n>Our results argue that latent world models should support not only prediction but reusable planning priors.
• Score: 25.440423602180278
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Latent world models are inherently strong encoders that transform image pixel to latent embedding, yet existing world models still rely on online trajectory optimization for action planning: for every state-goal pair, an iterative optimizer is run from scratch to search for optimal action sequences, treating the world model as a black-box simulator. This approach pays the full iterative optimization cost anew at every replanning step and reuses no planning experience across queries. In this work, we ask whether planning itself can be amortized once a latent world model has been learned. We present LeFlow, which learns a reusable latent trajectory prior operating directly in the latent dynamics space from the world model. LeFlow recasts planning as conditional latent trajectory generation: a rectified-flow model imagines a future latent path between the current and goal embeddings, an inverse dynamics decoder turns latent transitions into action chunks, and the frozen world model verifies each candidate by autoregressive rollout. Across four major goal-conditioned pixel-control benchmarks, LeFlow replaces iterative action-space optimization with amortized latent planning and fixed-budget rollout selection, achieving consistent success-rate gains with an order-of-magnitude reduction in planning time. Our results argue that latent world models should support not only prediction but reusable planning priors. Our code is available at https://github.com/hsiangwei0903/LeFlow.

Related papers

• Flow Matching in Feature Space for Stochastic World Modeling [21.159664319255665]
  We propose FlowWM, a world model that performs flow matching directly within pretrained feature space.<n>This is challenging because pretrained features are substantially high-dimensional, making standard diffusion recipes suboptimal.<n>We evaluate FlowWM on two benchmarks: a synthetic benchmark for systematic evaluation of accuracy and diversity, and a real-world benchmark FuturePerception.
  arXiv  Detail & Related papers  (2026-06-27T19:35:27Z)
• Beyond the Next Step: Variable-Length Latent World Models for Long-Horizon Planning [40.09669638091686]
  We propose a framework that learns to predict future latent states conditioned on action sequences of variable lengths.<n>Experiments show that Variable-length Latent World Models significantly improve latent space world models.
  arXiv  Detail & Related papers  (2026-06-19T21:50:15Z)
• Hierarchical Planning with Latent World Models [49.82095442483551]
  hierarchical planning achieves higher success while requiring up to 4x less planning-time compute.<n>We demonstrate that this hierarchical approach enables zero-shot control on real-world non-greedy robotic tasks.
  arXiv  Detail & Related papers  (2026-04-03T17:32:36Z)
• Uni-World VLA: Interleaved World Modeling and Planning for Autonomous Driving [52.04950569530877]
  We present Uni-World VLA, a unified vision-language-action model that tightly interleaves future frame prediction and trajectory planning.<n>Our approach achieves competitive closed-loop planning performance while producing high-fidelity future frame predictions.
  arXiv  Detail & Related papers  (2026-03-28T14:39:51Z)
• DynFlowDrive: Flow-Based Dynamic World Modeling for Autonomous Driving [67.83404465171257]
  DynFlowDrive is a latent world model that models the transition of world states under different driving actions.<n>It learns a velocity field that describes how the scene state changes under different driving actions.<n>Building upon this, we introduce a stability-aware multi-mode trajectory selection strategy.
  arXiv  Detail & Related papers  (2026-03-20T06:19:31Z)
• Parallel Stochastic Gradient-Based Planning for World Models [39.699893143984916]
  We propose a robust and highly parallelizable planner that leverages the differentiability of the learned world model for efficient optimization.<n>Our method treats states as optimization variables ("virtual states") with soft dynamics constraints, enabling parallel and easier optimization.<n>Our planner, which we call GRASP (GradAxed Planner), can be viewed as a valid version of a non-condensed or collocation-based optimal controller.
  arXiv  Detail & Related papers  (2026-01-31T02:57:47Z)
• Closing the Train-Test Gap in World Models for Gradient-Based Planning [64.36544881136405]
  We propose improved methods for training world models that enable efficient gradient-based planning.<n>At test time, our approach outperforms or matches the classical gradient-free cross-entropy method.
  arXiv  Detail & Related papers  (2025-12-10T18:59:45Z)
• Map-World: Masked Action planning and Path-Integral World Model for Autonomous Driving [35.521279875146526]
  Masked Action Planning (MAP) module treats future ego motion as masked sequence completion.<n>A lightweight world model rolls out future BEV semantics conditioned on each candidate trajectory.<n>On NAVSIM, our method matches anchor-based approaches and achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-11-25T10:30:26Z)
• WorldPrediction: A Benchmark for High-level World Modeling and Long-horizon Procedural Planning [52.36434784963598]
  We introduce WorldPrediction, a video-based benchmark for evaluating world modeling and procedural planning capabilities of different AI models.<n>We show that current frontier models barely achieve 57% accuracy on WorldPrediction-WM and 38% on WorldPrediction-PP whereas humans are able to solve both tasks perfectly.
  arXiv  Detail & Related papers  (2025-06-04T18:22:40Z)
• Enter the Void - Planning to Seek Entropy When Reward is Scarce [6.208654695856247]
  We propose a novel approach that anticipates and actively seeks out high-entropy states using short-horizon latent predictions.<n>We present a hierarchical planner that dynamically decides when to replan, planning horizon length, and the weighting between reward and entropy.<n>Our method finishes the Miniworld procedurally generated mazes 50% faster than base Dreamer at convergence and the policy trained in imagination converges in only 60% of the environment steps that base Dreamer needs.
  arXiv  Detail & Related papers  (2025-05-22T15:28:50Z)
• Learning from Reward-Free Offline Data: A Case for Planning with Latent Dynamics Models [79.2162092822111]
  We systematically evaluate reinforcement learning (RL) and control-based methods on a suite of navigation tasks.<n>We employ a latent dynamics model using the Joint Embedding Predictive Architecture (JEPA) and employ it for planning.<n>Our results show that model-free RL benefits most from large amounts of high-quality data, whereas model-based planning generalizes better to unseen layouts.
  arXiv  Detail & Related papers  (2025-02-20T18:39:41Z)
• Autoregressive Dynamics Models for Offline Policy Evaluation and Optimization [60.73540999409032]
  We show that expressive autoregressive dynamics models generate different dimensions of the next state and reward sequentially conditioned on previous dimensions. We also show that autoregressive dynamics models are useful for offline policy optimization by serving as a way to enrich the replay buffer.
  arXiv  Detail & Related papers  (2021-04-28T16:48:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.