FlowMPC: Improving Flow Matching policies with World Models

• URL: http://arxiv.org/abs/2606.16286v1
• Date: Mon, 15 Jun 2026 06:50:11 GMT
• Title: FlowMPC: Improving Flow Matching policies with World Models
• Abstract summary: Flow Matching (FM) is a powerful approach for behavior cloning in multimodal action spaces.<n>Because it is not trained to directly maximize expected return, there is still room to improve how FM policies act at test time.<n>This work investigates whether a learned world model can improve FM policies by enabling Model Predictive Path Integral (MPPI) planning over candidate action sequences proposed by the policy.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Flow Matching (FM) is a powerful approach for behavior cloning in multimodal action spaces [Jiang et al., 2025], but because it is not trained to directly maximize expected return, there is still room to improve how FM policies act at test time. This work investigates whether a learned world model can improve FM policies by enabling Model Predictive Path Integral (MPPI) planning over candidate action sequences proposed by the policy. Building on TD-MPC2 [Hansen et al., 2024], I introduce FlowMPC, a framework that combines an imitation-learned FM policy with a learned world model for test-time planning in ManiSkill manipulation tasks [Tao et al., 2025]. Across PickCube and PickSingleYCB, adding the world model improved performance over the FM policy alone, with especially clear gains in end-of-episode success. These results suggest that world-model-based planning can effectively complement flow-based imitation policies without modifying the FM training objective.

Related papers

• MODIP: Efficient Model-Based Optimization for Diffusion Policies [13.033386079956266]
  Diffusion policies (DPs) have emerged as expressive policy representations for robot learning.<n>We propose MODIP, a framework for the offline-to-online fine-tuning of DPs.
  arXiv  Detail & Related papers  (2026-06-09T13:09:21Z)
• Scaling World-Model Reinforcement Learning Through Diffusion Policy Optimization [46.32504081845328]
  Model-based reinforcement learning can be effectively supported at scale through the use of world models.<n>We identify a bottleneck: a structural misalignment between search and value learning in existing world model approaches.<n>We propose Model-Based Diffusion Policy Optimization (MBDPO) in world models, a framework that unifies search and policy optimization through diffusion policy representations.
  arXiv  Detail & Related papers  (2026-05-25T19:06:51Z)
• World Action Verifier: Self-Improving World Models via Forward-Inverse Asymmetry [82.93104394404781]
  General-purpose world models promise scalable policy evaluation, optimization, and planning.<n>We propose World Action Verifier (WAV), a framework that enables world models to identify their own prediction errors and self-improve.
  arXiv  Detail & Related papers  (2026-04-02T12:48:36Z)
• Enhancing Policy Learning with World-Action Model [4.941630596191806]
  World-Action Model (WAM) is an action-regularized world model that jointly reasons over future visual observations and the actions that drive state transitions.<n>WAM incorporates an inverse dynamics objective into DreamerV2 that predicts actions from latent state transitions.<n>We evaluate WAM on enhancing policy learning across eight manipulation tasks from the CALVIN benchmark.
  arXiv  Detail & Related papers  (2026-03-30T19:56:05Z)
• Bootstrap Off-policy with World Model [59.129118672069644]
  We propose BOOM, a framework that tightly integrates planning and off-policy learning through a bootstrap loop.<n>BOOM achieves state-of-the-art results in both training stability and final performance.
  arXiv  Detail & Related papers  (2025-11-01T06:33:04Z)
• A KL-regularization framework for learning to plan with adaptive priors [1.0246259631050245]
  We introduce Policy Optimization-Model Predictive Control (PO-MPC)<n>PO-MPC integrates the planner's action distribution as a prior in policy optimization.<n>Our experiments show that these extended configurations yield significant performance improvements.
  arXiv  Detail & Related papers  (2025-10-05T16:45:38Z)
• Flow Matching Policy Gradients [33.011978450108636]
  Flow Policy Optimization casts policy optimization as maximizing an advantage-weighted ratio computed from the conditional flow matching loss.<n>We show that FPO can train diffusion-style policies from scratch in a variety of continuous control tasks.<n>We find that flow-based models can capture multimodal action distributions and achieve higher performance than Gaussian policies, particularly in under-conditioned settings.
  arXiv  Detail & Related papers  (2025-07-28T17:59:57Z)
• Decision Flow Policy Optimization [53.825268058199825]
  We show that generative models can effectively model complex multi-modal action distributions and achieve superior robotic control in continuous action spaces.<n>Previous methods usually adopt the generative models as behavior models to fit state-conditioned action distributions from datasets.<n>We propose Decision Flow, a unified framework that integrates multi-modal action distribution modeling and policy optimization.
  arXiv  Detail & Related papers  (2025-05-26T03:42:20Z)
• PWM: Policy Learning with Multi-Task World Models [37.678858748473196]
  World model methods offer scalability by learning a simulation of the environment.<n> gradient-based methods exhibit lower variance but fail to handle discontinuities.<n>We introduce Policy learning with multi-task World Models (PWM), a novel model-based RL algorithm for continuous control.
  arXiv  Detail & Related papers  (2024-07-02T17:47:03Z)
• Theoretically Guaranteed Policy Improvement Distilled from Model-Based Planning [64.10794426777493]
  Model-based reinforcement learning (RL) has demonstrated remarkable successes on a range of continuous control tasks. Recent practices tend to distill optimized action sequences into an RL policy during the training phase. We develop an approach to distill from model-based planning to the policy.
  arXiv  Detail & Related papers  (2023-07-24T16:52:31Z)
• Evaluating model-based planning and planner amortization for continuous control [79.49319308600228]
  We take a hybrid approach, combining model predictive control (MPC) with a learned model and model-free policy learning. We find that well-tuned model-free agents are strong baselines even for high DoF control problems. We show that it is possible to distil a model-based planner into a policy that amortizes the planning without any loss of performance.
  arXiv  Detail & Related papers  (2021-10-07T12:00:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.