FlowR2A: Learning Reward-to-Action Distribution for Multimodal Driving Planning

• URL: http://arxiv.org/abs/2606.24231v1
• Date: Tue, 23 Jun 2026 07:21:36 GMT
• Title: FlowR2A: Learning Reward-to-Action Distribution for Multimodal Driving Planning
• Abstract summary: FlowR2A unifies the dense supervision of scoring-based methods with the proposal generation of anchor-based methods in a single generative model.<n>To balance hard safety constraints against soft progress objectives, we introduce fine-grained per-timestep reward conditioning and reward noise augmentation.<n>The generative formulation naturally supports controllable test-time sampling via reward guidance and anchored sampling, producing high-quality proposals.
• Score: 63.80677039120727
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multimodal driving planning faces a long-standing tension between two paradigms: scoring-based methods benefit from dense reward supervision but are confined to a fixed action vocabulary, while anchor-based methods generate proposals dynamically yet suffer from sparse supervision constrained to a single ground-truth trajectory. In this work, we propose FlowR2A, which resolves this tension by reframing simulation-based rewards from discriminative targets into generative conditions. By learning the reward-conditioned action distribution from dense trajectory-reward pairs with a flow-matching decoder, FlowR2A unifies the dense supervision of scoring-based methods with the proposal generation of anchor-based methods in a single generative model, forcing the model to internalize the correlation between an action and its outcomes in safety, progress, comfort, and rule compliance. To balance hard safety constraints against soft progress objectives, we introduce fine-grained per-timestep reward conditioning and reward noise augmentation. The generative formulation naturally supports controllable test-time sampling via reward guidance and anchored sampling, producing high-quality proposals. FlowR2A achieves state-of-the-art results on the NAVSIM v1 and v2 benchmarks, with multimodal proposals of substantially higher quality than prior methods.

Related papers

• Motion Planning with Model-Based Diffusion via Constraint Optimization and Adaptive Scheduling [7.761673975696802]
  Single-Robot Motion Planning (SRMP) is challenging under complex constraints and computational limits.<n>Recent Model-Based Diffusion (MBD) recasts the SRMP to guide samples from a posterior diffusion trajectories.<n>Our method achieves higher safety & success rates, faster convergence, and lower final costs than baseline planners.
  arXiv  Detail & Related papers  (2026-07-16T01:04:32Z)
• Dual-Flow Reinforcement Learning with State-Aware Exploration [16.944702024760094]
  In continuous-control reinforcement learning tasks, multimodal optimal actions often coincide with uncertain, multimodal return distributions.<n>We propose Dual-Flow RL, a unified actor-critic framework that jointly models a continuous return distribution and a multimodal policy distribution.<n>Experiments on DeepMind Control Suite and Humanoid-Bench show that Dual-Flow RL achieves state-of-the-art performance on most tasks.
  arXiv  Detail & Related papers  (2026-06-29T05:57:33Z)
• Generative Actor-Critic with Soft Bridge Policies [19.121990264725028]
  Training effective soft generative policies faces two obstacles that often arise together.<n>First, marginal action densities are often unavailable, so existing methods typically rely on entropy bounds, proxies or approximations.<n>These obstacles motivate us to seek a generative policy that exposes a tractable MaxEnt objective while requiring only a single sampled actor forward pass for action generation.
  arXiv  Detail & Related papers  (2026-05-09T06:36:32Z)
• FAVE: Flow-based Average Velocity Establishment for Sequential Recommendation [33.3202615024807]
  We propose a Flow-based Average Velocity Establishment (Fave) framework for one-step generation recommendation.<n>Fave learns a direct trajectory from an informative prior to the target distribution.<n>Experiments on three benchmarks demonstrate that Fave not only achieves state-of-the-art recommendation performance but also delivers an order-of-magnitude improvement in inference efficiency.
  arXiv  Detail & Related papers  (2026-04-06T05:17:43Z)
• Primary-Fine Decoupling for Action Generation in Robotic Imitation [91.2899765310853]
  Multi-modal distribution in robotic manipulation action sequences poses critical challenges for imitation learning.<n>We propose Primary-Fine Decoupling for Action Generation (PF-DAG), a two-stage framework that decouples coarse action consistency from fine-grained variations.<n>PF-DAG outperforms state-of-the-art baselines across 56 tasks from Adroit, DexArt, and MetaWorld benchmarks.
  arXiv  Detail & Related papers  (2026-02-25T08:36:45Z)
• OmniVL-Guard: Towards Unified Vision-Language Forgery Detection and Grounding via Balanced RL [63.388513841293616]
  Existing forgery detection methods fail to handle the interleaved text, images, and videos prevalent in real-world misinformation.<n>To bridge this gap, this paper targets to develop a unified framework for omnibus vision-language forgery detection and grounding.<n>We propose textbf OmniVL-Guard, a balanced reinforcement learning framework for omnibus vision-language forgery detection and grounding.
  arXiv  Detail & Related papers  (2026-02-11T09:41:36Z)
• A Unified Density Operator View of Flow Control and Merging [37.902481322917396]
  We introduce a unifying probability-space framework that subsumes both as limit cases, and enables reward-guided flow merging.<n>We also introduce Reward-Guided Flow Merging (RFM), a mirror-descent scheme that reduces reward-guided flow merging to a sequence of standard fine-tuning problems.<n>We provide first-of-their-kind theoretical guarantees for reward-guided and pure flow merging via RFM.
  arXiv  Detail & Related papers  (2026-02-08T15:27:28Z)
• Euphonium: Steering Video Flow Matching via Process Reward Gradient Guided Stochastic Dynamics [49.242224984144904]
  We propose Euphonium, a novel framework that steers generation via process reward gradient guided dynamics.<n>Our key insight is to formulate the sampling process as a theoretically principled algorithm that explicitly incorporates the gradient of a Process Reward Model.<n>We derive a distillation objective that internalizes the guidance signal into the flow network, eliminating inference-time dependency on the reward model.
  arXiv  Detail & Related papers  (2026-02-04T08:59:57Z)
• DenseGRPO: From Sparse to Dense Reward for Flow Matching Model Alignment [49.45064510462232]
  GRPO-based approaches for text-to-image generation suffer from the sparse reward problem.<n>We introduce textbfDenseGRPO, a novel framework that aligns human preference with dense rewards.
  arXiv  Detail & Related papers  (2026-01-28T03:39:05Z)
• Generative Actor Critic [74.04971271003869]
  Generative Actor Critic (GAC) is a novel framework that decouples sequential decision-making by reframing textitpolicy evaluation as learning a generative model of the joint distribution over trajectories and returns.<n>Experiments on Gym-MuJoCo and Maze2D benchmarks demonstrate GAC's strong offline performance and significantly enhanced offline-to-online improvement compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-12-25T06:31:11Z)
• G$^2$RPO: Granular GRPO for Precise Reward in Flow Models [74.21206048155669]
  We propose a novel Granular-GRPO (G$2$RPO) framework that achieves precise and comprehensive reward assessments of sampling directions.<n>We introduce a Multi-Granularity Advantage Integration module that aggregates advantages computed at multiple diffusion scales.<n>Our G$2$RPO significantly outperforms existing flow-based GRPO baselines.
  arXiv  Detail & Related papers  (2025-10-02T12:57:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.