Mean Flow Policy with Instantaneous Velocity Constraint for One-step Action Generation

• URL: http://arxiv.org/abs/2602.13810v1
• Date: Sat, 14 Feb 2026 14:44:06 GMT
• Title: Mean Flow Policy with Instantaneous Velocity Constraint for One-step Action Generation
• Authors: Guojian Zhan, Letian Tao, Pengcheng Wang, Yixiao Wang, Yiheng Li, Yuxin Chen, Masayoshi Tomizuka, Shengbo Eben Li,
• Abstract summary: Mean velocity policy (MVP) is a new generative policy function that models the mean velocity field to achieve the fastest one-step action generation.<n>MVP achieves state-of-the-art success rates across several challenging robotic manipulation tasks from Robomimic and OGBench.
• Score: 65.13627721310613
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Learning expressive and efficient policy functions is a promising direction in reinforcement learning (RL). While flow-based policies have recently proven effective in modeling complex action distributions with a fast deterministic sampling process, they still face a trade-off between expressiveness and computational burden, which is typically controlled by the number of flow steps. In this work, we propose mean velocity policy (MVP), a new generative policy function that models the mean velocity field to achieve the fastest one-step action generation. To ensure its high expressiveness, an instantaneous velocity constraint (IVC) is introduced on the mean velocity field during training. We theoretically prove that this design explicitly serves as a crucial boundary condition, thereby improving learning accuracy and enhancing policy expressiveness. Empirically, our MVP achieves state-of-the-art success rates across several challenging robotic manipulation tasks from Robomimic and OGBench. It also delivers substantial improvements in training and inference speed over existing flow-based policy baselines.

Related papers

• SpeedAug: Policy Acceleration via Tempo-Enriched Policy and RL Fine-Tuning [52.29534291796025]
  Reinforcement learning is a promising approach that adapts policies for faster execution without additional demonstration.<n>We propose SpeedAug, an RL-based policy acceleration framework that efficiently adapts pre-trained policies for faster task execution.
  arXiv  Detail & Related papers  (2025-11-24T04:25:47Z)
• One-Step Generative Policies with Q-Learning: A Reformulation of MeanFlow [56.13949180229929]
  We introduce a one-step generative policy for offline reinforcement learning that maps noise directly to actions via a residual reformulation of MeanFlow.<n>Our method achieves strong performance in both offline and offline-to-online reinforcement learning settings.
  arXiv  Detail & Related papers  (2025-11-17T06:34:17Z)
• Iterative Refinement of Flow Policies in Probability Space for Online Reinforcement Learning [56.47948583452555]
  We introduce the Stepwise Flow Policy (SWFP) framework, founded on the key insight that discretizing the flow matching inference process via a fixed-step Euler scheme aligns it with the variational Jordan-Kinderlehrer-Otto principle from optimal transport.<n>SWFP decomposes the global flow into a sequence of small, incremental transformations between proximate distributions.<n>This decomposition yields an efficient algorithm that fine-tunes pre-trained flows via a cascade of small flow blocks, offering significant advantages.
  arXiv  Detail & Related papers  (2025-10-17T07:43:51Z)
• SAC Flow: Sample-Efficient Reinforcement Learning of Flow-Based Policies via Velocity-Reparameterized Sequential Modeling [9.936731043466699]
  Training expressive flow-based policies with off-policy reinforcement learning is notoriously unstable due to gradient pathologies in the multi-step action sampling process.<n>We trace this instability to a fundamental connection: the flow rollout is algebraically equivalent to a residual recurrent computation, making it susceptible to the same vanishing and exploding gradients as RNNs.<n>We develop a practical SAC-based algorithm, enabled by a noise-augmented rollout, that facilitates direct end-to-end training of these policies.
  arXiv  Detail & Related papers  (2025-09-30T04:21:20Z)
• FastGRPO: Accelerating Policy Optimization via Concurrency-aware Speculative Decoding and Online Draft Learning [11.68914161151634]
  Group relative policy optimization (GRPO) has demonstrated significant potential in improving the reasoning capabilities of large language models.<n>We propose a speculative decoding framework that adjusts the drafting and verification strategy according to real-time levels.<n>We show that the proposed method achieves end-to-end speedups of 2.35x to 2.72x, significantly surpassing baseline approaches in efficiency.
  arXiv  Detail & Related papers  (2025-09-26T02:48:41Z)
• One-Step Flow Policy Mirror Descent [52.31612487608593]
  Flow Policy Mirror Descent (FPMD) is an online RL algorithm that enables 1-step sampling during flow policy inference.<n>Our approach exploits a theoretical connection between the distribution variance and the discretization error of single-step sampling in straight flow matching models.
  arXiv  Detail & Related papers  (2025-07-31T15:51:10Z)
• Flow-Based Policy for Online Reinforcement Learning [34.86742824686496]
  FlowRL is a framework for online reinforcement learning that integrates flow-based policy representation with Wasserstein-2-regularized optimization.<n>We show that FlowRL achieves competitive performance in online reinforcement learning benchmarks.
  arXiv  Detail & Related papers  (2025-06-15T10:53:35Z)
• Fast Adaptation with Behavioral Foundation Models [82.34700481726951]
  Unsupervised zero-shot reinforcement learning has emerged as a powerful paradigm for pretraining behavioral foundation models.<n>Despite promising results, zero-shot policies are often suboptimal due to errors induced by the unsupervised training process.<n>We propose fast adaptation strategies that search in the low-dimensional task-embedding space of the pre-trained BFM to rapidly improve the performance of its zero-shot policies.
  arXiv  Detail & Related papers  (2025-04-10T16:14:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.