Flow-Map GRPO: Reinforcement Learning for Few-Step Flow-Map Generators via Anchored Stochastic Composition

• URL: http://arxiv.org/abs/2607.00535v1
• Date: Wed, 01 Jul 2026 07:25:11 GMT
• Title: Flow-Map GRPO: Reinforcement Learning for Few-Step Flow-Map Generators via Anchored Stochastic Composition
• Abstract summary: Few-step flow-map generators, such as consistency models and MeanFlow, accelerate sampling by directly learning longrange transport maps between noise and data.<n>These models are typically deterministic, which makes them difficult to optimize with post-training methods.<n>We propose Flow-MapO, an online framework for deterministic post-training framework for fewstep flow-map generators.
• Score: 13.766141027420879
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Few-step flow-map generators, such as consistency models and MeanFlow, accelerate sampling by directly learning long-range transport maps between noise and data. However, these models are typically deterministic, which makes them difficult to optimize with reinforcement learning (RL) post-training methods that require stochastic trajectories and well-defined likelihood ratios. Existing SDE-based stochasticization techniques are designed for velocity-based samplers with infinitesimal or finely discretized transitions, and therefore do not directly apply to long-range flow maps. In this work, we propose Flow-Map GRPO, an online RL post-training framework for deterministic few-step flow-map generators. The key component is Anchored Stochastic Flow Map Composition (ASFMC), a path-preserving stochasticization mechanism that introduces randomness through anchor-based conditional resampling while preserving the original marginal probability path of the deterministic flow map. We derive GRPO objectives for both single-time and two-time flow-map parameterizations. Experiments on few-step FLUX-based text-to-image generators, including MeanFlow and sCM, show that Flow-Map GRPO improves pretrained deterministic flow-map models across reward-based, perceptual, and task-level evaluation metrics. Our results demonstrate that deterministic few-step flow-map generators can be effectively aligned with RL post-training without modifying their original model parameterization or retraining them as native stochastic models.

Related papers

• Flow Map Learning via Nongradient Vector Flow [22.985676782036904]
  Diffusion and flow-based models benefit from simple regression losses, but inference incurs significant overhead because sampling requires integration.<n>We introduce SGFlow, an approach for learning flow maps that bypasses explicit invertibility constraints and expensive differentiation through model iteration.
  arXiv  Detail & Related papers  (2026-07-29T02:17:27Z)
• Itô maps for any-step SDEs [5.492715250025273]
  Recent one-step generative models accelerate sampling by learning deterministic flow maps of underlying dynamics.<n>We introduce the It map, an any-step flow map that takes an intermediate state and Brownian path and predicts future states in a single pass.<n>It maps produce diverse, conditionally valid endpoint samples from fixed intermediate states and support strong steering performance on synthetic and image-generation benchmarks.
  arXiv  Detail & Related papers  (2026-06-09T17:39:22Z)
• Strong Stochastic Flow Maps [38.75021352054866]
  Flow and diffusion models generate high-quality samples in many modalities.<n> Flow maps alleviate this problem by learning the solution map of the differential equation directly.<n>We propose Strong Flow Maps (SSFMs) as a novel framework for learning the strong solution map of additive-noise SDEs.
  arXiv  Detail & Related papers  (2026-05-31T08:14:47Z)
• Entropy Across the Bridge: Conditional-Marginal Discretization for Flow and Schrödinger Samplers [68.9946633641494]
  Flow matching and Schrdinger bridges define probability, yet their inference grids are usually or inherited from one-endpoint diffusion.<n>We derive a conditional-marginal entropy-rate objective for bridge-aware discretization, separating endpoint-conditioned bridge geometry from marginal flow evolution.<n>On EDM/CIFAR-10, the entropic time-discretization gives the best tested five-step FID (186.3 pm 4.0 versus 200.5 pm 2.9 for linear and 238.0 pm 5.3 for cosine)
  arXiv  Detail & Related papers  (2026-05-15T16:11:10Z)
• AnyFlow: Any-Step Video Diffusion Model with On-Policy Flow Map Distillation [70.80207977907908]
  We introduce AnyFlow, the first any-step video diffusion distillation framework based on flow maps.<n>We show that AnyFlow achieves performance matches or surpasses consistency-based counterparts in the few-step regime.
  arXiv  Detail & Related papers  (2026-05-13T16:06:34Z)
• One-Step Generative Modeling via Wasserstein Gradient Flows [56.266175641614986]
  W-Flow is a framework for training a generator that transforms samples from a simple reference distribution into samples from a target data distribution in a single step.<n>We set a new state of the art for one-step ImageNet 256$times$256 generation, achieving 1.29 FID, with improved mode coverage and domain transfer.
  arXiv  Detail & Related papers  (2026-05-12T08:29:44Z)
• Riemannian MeanFlow for One-Step Generation on Manifolds [54.09734511705173]
  Flow Matching enables simulation-free training of generative models.<n>MeanFlow can be extended to manifold-valued generation where velocities lie in location-dependent tangent spaces.<n>Experiments on spheres, tori, and SO(3) demonstrate competitive one-step sampling with improved quality-efficiency trade-offs and substantially reduced sampling cost.
  arXiv  Detail & Related papers  (2026-03-11T12:41:46Z)
• FlowConsist: Make Your Flow Consistent with Real Trajectory [99.22869983378062]
  We argue that current fast-flow training paradigms suffer from two fundamental issues.<n> conditional velocities constructed from randomly paired noise-data samples introduce systematic trajectory drift.<n>We propose FlowConsist, a training framework designed to enforce trajectory consistency in fast flows.
  arXiv  Detail & Related papers  (2026-02-06T03:24:23Z)
• Test-time scaling of diffusions with flow maps [68.79792714591564]
  A common recipe to improve diffusion models at test-time is to introduce the gradient of the reward into the dynamics of the diffusion itself.<n>We propose a simple solution by working directly with a flow map.<n>By exploiting a relationship between the flow map and velocity field governing the instantaneous transport, we construct an algorithm, Flow Map Trajectory Tilting (FMTT), which provably performs better ascent on the reward than standard test-time methods.
  arXiv  Detail & Related papers  (2025-11-27T18:44:12Z)
• Flow map matching with stochastic interpolants: A mathematical framework for consistency models [15.520853806024943]
  Flow Map Matching is a principled framework for learning the two-time flow map of an underlying generative model.<n>We show that FMM unifies and extends a broad class of existing approaches for fast sampling.
  arXiv  Detail & Related papers  (2024-06-11T17:41:26Z)
• Learning Stochastic Dynamical System via Flow Map Operator [3.9752110899603053]
  We present a numerical framework for learning unknown dynamical systems using measurement data. For learning systems, we define a flow map that is a superposition of two sub-flow maps. The training data are used to construct the deterministic sub-map first, followed by the sub-map.
  arXiv  Detail & Related papers  (2023-05-05T23:24:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.