ScoRe-Flow: Complete Distributional Control via Score-Based Reinforcement Learning for Flow Matching

• URL: http://arxiv.org/abs/2604.10962v1
• Date: Mon, 13 Apr 2026 03:56:37 GMT
• Title: ScoRe-Flow: Complete Distributional Control via Score-Based Reinforcement Learning for Flow Matching
• Abstract summary: Flow Matching (FM) policies have emerged as an efficient backbone for robotic control.<n>We propose ScoRe-Flow, a score-based RL fine-tuning method to achieve decoupled control over the mean and variance of transitions.
• Score: 14.421249343302028
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Flow Matching (FM) policies have emerged as an efficient backbone for robotic control, offering fast and expressive action generation that underpins recent large-scale embodied AI systems. However, FM policies trained via imitation learning inherit the limitations of demonstration data; surpassing suboptimal behaviors requires reinforcement learning (RL) fine-tuning. Recent methods convert deterministic flows into stochastic differential equations (SDEs) with learnable noise injection, enabling exploration and tractable likelihoods, but such noise-only control can compromise training efficiency when demonstrations already provide strong priors. We observe that modulating the drift via the score function, i.e., the gradient of log-density, steers exploration toward high-probability regions, improving stability. The score admits a closed-form expression from the velocity field, requiring no auxiliary networks. Based on this, we propose ScoRe-Flow, a score-based RL fine-tuning method that combines drift modulation with learned variance prediction to achieve decoupled control over the mean and variance of stochastic transitions. Experiments demonstrate that ScoRe-Flow achieves 2.4x faster convergence than flow-based SOTA on D4RL locomotion tasks and up to 5.4% higher success rates on Robomimic and Franka Kitchen manipulation tasks.

Related papers

• StructRL: Structured Action-Space Exploration for Flow-Based VLAs [81.43458629514139]
  Flow-based Vision-Language-Action models are now widely used for continuous robotic manipulation.<n>Online reinforcement learning is emerging as a key technique for adapting them to new tasks.<n>We show that effective robot exploration calls for structured noise: temporally smooth and scaled differently across action groups.
  arXiv  Detail & Related papers  (2026-08-15T09:31:29Z)
• Boosting Neural Video Codec via Scale-Driven Online Flow Refinement [59.68390983151205]
  We propose a Training-Free Scale-Driven Online Flow Refinement (SOFR) method.<n>It fuses motion information from coarse and fine scales according to warping accuracy, effectively rectifying motion estimation errors.<n>It brings an average of 2.84% and 4.05% savings in terms of PSNR and MS-SSIM, respectively.
  arXiv  Detail & Related papers  (2026-06-22T08:36:23Z)
• ProbeFlow: Training-Free Adaptive Flow Matching for Vision-Language-Action Models [12.232846303286161]
  Recent Vision-Language-Action (VLA) models equipped with Flow Matching (FM) action heads achieve state-of-the-art performance in complex robot manipulation.<n>While current acceleration efforts optimize the Vision-Language Model (VLM) backbone, the action head bottleneck remains overlooked.<n>We propose ProbeFlow, a training-free adaptive inference framework for continuous robotic control.
  arXiv  Detail & Related papers  (2026-03-18T15:38:29Z)
• MeanFuser: Fast One-Step Multi-Modal Trajectory Generation and Adaptive Reconstruction via MeanFlow for End-to-End Autonomous Driving [23.013043338076745]
  MeanFuser is an end-to-end autonomous driving method.<n>We introduce GMN to guide generative sampling and adapt MeanFlow Identity" to end-to-end planning.<n>Experiments on the NAVSIM closed-loop benchmark demonstrate that MeanFuser achieves outstanding performance without the supervision of the PDM Score.
  arXiv  Detail & Related papers  (2026-02-23T17:17:26Z)
• RFS: Reinforcement Learning with Residual Flow Steering for Dexterous Manipulation [7.500999283386335]
  Residual Flow Steering (RFS) is a data-efficient reinforcement learning framework for adapting pretrained generative policies.<n>RFS steers a pretrained flow-matching policy by jointly optimizing a residual action and a latent noise distribution.<n>We demonstrate the effectiveness of RFS on dexterous manipulation tasks, showing efficient fine-tuning in both simulation and real-world settings.
  arXiv  Detail & Related papers  (2026-02-02T08:11:57Z)
• Steering Vision-Language-Action Models as Anti-Exploration: A Test-Time Scaling Approach [78.4812458793128]
  We propose textbfTACO, a test-time-scaling framework that applies a lightweight pseudo-count estimator as a high-fidelity verifier of action chunks.<n>Our method resembles the classical anti-exploration principle in offline reinforcement learning (RL), and being gradient-free, it incurs significant computational benefits.
  arXiv  Detail & Related papers  (2025-12-02T14:42:54Z)
• One-Step Generative Policies with Q-Learning: A Reformulation of MeanFlow [56.13949180229929]
  We introduce a one-step generative policy for offline reinforcement learning that maps noise directly to actions via a residual reformulation of MeanFlow.<n>Our method achieves strong performance in both offline and offline-to-online reinforcement learning settings.
  arXiv  Detail & Related papers  (2025-11-17T06:34:17Z)
• NoisyRollout: Reinforcing Visual Reasoning with Data Augmentation [66.36912000442608]
  NoisyRollout is a simple yet effective data augmentation method.<n>It mixes training trajectories from both clean and moderately distorted images.<n>It achieves state-of-the-art performance among open-source RL-tuned models.
  arXiv  Detail & Related papers  (2025-04-17T16:10:13Z)
• FlowTS: Time Series Generation via Rectified Flow [67.41208519939626]
  FlowTS is an ODE-based model that leverages rectified flow with straight-line transport in probability space.<n>For unconditional setting, FlowTS achieves state-of-the-art performance, with context FID scores of 0.019 and 0.011 on Stock and ETTh datasets.<n>For conditional setting, we have achieved superior performance in solar forecasting.
  arXiv  Detail & Related papers  (2024-11-12T03:03:23Z)
• Value function estimation using conditional diffusion models for control [62.27184818047923]
  We propose a simple algorithm called Diffused Value Function (DVF) It learns a joint multi-step model of the environment-robot interaction dynamics using a diffusion model. We show how DVF can be used to efficiently capture the state visitation measure for multiple controllers.
  arXiv  Detail & Related papers  (2023-06-09T18:40:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.