RAD-2: Scaling Reinforcement Learning in a Generator-Discriminator Framework

• URL: http://arxiv.org/abs/2604.15308v1
• Date: Thu, 16 Apr 2026 17:59:44 GMT
• Title: RAD-2: Scaling Reinforcement Learning in a Generator-Discriminator Framework
• Abstract summary: RAD-2 is a unified generator-discriminator framework for closed-loop planning.<n>It produces diverse trajectory candidates, while an RL-optimized discriminator reranks these candidates according to their long-term driving quality.<n>It reduces the collision rate by 56% compared with strong diffusion-based planners.
• Score: 49.531001563853984
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: High-level autonomous driving requires motion planners capable of modeling multimodal future uncertainties while remaining robust in closed-loop interactions. Although diffusion-based planners are effective at modeling complex trajectory distributions, they often suffer from stochastic instabilities and the lack of corrective negative feedback when trained purely with imitation learning. To address these issues, we propose RAD-2, a unified generator-discriminator framework for closed-loop planning. Specifically, a diffusion-based generator is used to produce diverse trajectory candidates, while an RL-optimized discriminator reranks these candidates according to their long-term driving quality. This decoupled design avoids directly applying sparse scalar rewards to the full high-dimensional trajectory space, thereby improving optimization stability. To further enhance reinforcement learning, we introduce Temporally Consistent Group Relative Policy Optimization, which exploits temporal coherence to alleviate the credit assignment problem. In addition, we propose On-policy Generator Optimization, which converts closed-loop feedback into structured longitudinal optimization signals and progressively shifts the generator toward high-reward trajectory manifolds. To support efficient large-scale training, we introduce BEV-Warp, a high-throughput simulation environment that performs closed-loop evaluation directly in Bird's-Eye View feature space via spatial warping. RAD-2 reduces the collision rate by 56% compared with strong diffusion-based planners. Real-world deployment further demonstrates improved perceived safety and driving smoothness in complex urban traffic.

Related papers

• One Diffusion Model, Two Roles: Guided Trajectory Planning and Safety-Critical Scenario Generation in Closed-Loop Simulation [4.570068778162319]
  We show that a single pretrained diffusion traffic model can serve two complementary roles in the autonomous driving development loop.<n>On the planning side, we introduce a Single-Stream Dual-Stream (SSDS) diffusion-transformer decoder that fuses scene context via joint attention.<n>We leverage the same diffusion model as a controllable scenario generator to create realistic long-tail driving interactions for closed-loop evaluation.
  arXiv  Detail & Related papers  (2026-09-04T09:22:54Z)
• G2DP: Diffusion Planning with Spatio-Temporal Grid Guidance [44.0166619948181]
  In autonomous driving, diffusion-based planners have emerged as a promising paradigm for robust motion planning in dense and interactive traffic.<n>Existing guidance typically relies on sparse, entity-centric geometric queries or post-hoc refinement, yielding limited awareness and fragile performance in interactive scenes.<n>G2DP (GuidedGrid-Guided Planning) constructs a differentiable-temporal cost volume by fusing probabilistic future occupancy guidance with a route-time map.<n>It achieves state-of-the-art performance on nuPlan, outperforming the strongest imitation-learning baseline by +7.2 points in reactive transfers.
  arXiv  Detail & Related papers  (2026-06-24T16:38:55Z)
• MAPLE: Latent Multi-Agent Play for End-to-End Autonomous Driving [62.43744546817599]
  Vision-language--action (VLA) models are effective as end-to-end motion planners, but can be brittle when evaluated in closed-loop settings.<n>We propose MAPLE, a novel framework for reactive, multi-agent rollout of a dynamic driving scenario in the latent space of the VLA model.<n> MAPLE achieves state-of-the-art driving performance on Bench2Drive and demonstrates scalable, closed-loop multi-agent play for robust E2E autonomous driving systems.
  arXiv  Detail & Related papers  (2026-05-13T23:35:14Z)
• HAD: Combining Hierarchical Diffusion with Metric-Decoupled RL for End-to-End Driving [51.268878540511054]
  We propose HAD, an end-to-end planning framework with a Hierarchical Diffusion Policy.<n>We show that HAD achieves new state-of-the-art performance on both NAVSIM and HUGSIM.
  arXiv  Detail & Related papers  (2026-04-04T04:12:47Z)
• Amortizing Trajectory Diffusion with Keyed Drift Fields [2.5352713493505785]
  Keyed Drifting Policies is a one-step trajectory generator trained with a drift-field objective.<n>It repels generated trajectories toward condition-matched dataset windows.<n>It achieves strong performance with one-step inference and substantially lower planning latency than diffusion sampling.
  arXiv  Detail & Related papers  (2026-03-14T17:56:15Z)
• VORL-EXPLORE: A Hybrid Learning Planning Approach to Multi-Robot Exploration in Dynamic Environments [19.811711277536187]
  Multi-robot exploration commonly decouples frontier allocation from local navigation.<n>We propose VORL-EXPLORE, a hybrid learning and planning framework that addresses this limitation through execution fidelity.<n>It also drives a risk-aware adaptive arbitration mechanism between global A* guidance and a reactive reinforcement learning policy.
  arXiv  Detail & Related papers  (2026-03-09T05:20:33Z)
• PlannerRFT: Reinforcing Diffusion Planners through Closed-Loop and Sample-Efficient Fine-Tuning [17.21182051618103]
  Diffusion-based planners have emerged as a promising approach for human-like trajectory generation in autonomous driving.<n>Recent works incorporate reinforcement fine-tuning to enhance the robustness of diffusion planners.<n>We propose PlannerRFT, a sample-efficient reinforcement fine-tuning framework for diffusion-based planners.
  arXiv  Detail & Related papers  (2026-01-19T09:56:05Z)
• DiffusionDriveV2: Reinforcement Learning-Constrained Truncated Diffusion Modeling in End-to-End Autonomous Driving [65.7087560656003]
  Generative diffusion models for end-to-end autonomous driving often suffer from mode collapse.<n>We propose DiffusionDriveV2, which leverages reinforcement learning to constrain low-quality modes and explore for superior trajectories.<n>This significantly enhances the overall output quality while preserving the inherent multimodality of its core Gaussian Mixture Model.
  arXiv  Detail & Related papers  (2025-12-08T17:29:52Z)
• Discrete-Guided Diffusion for Scalable and Safe Multi-Robot Motion Planning [56.240199425429445]
  Multi-Robot Motion Planning (MPMP) involves generating trajectories for multiple robots operating in a shared continuous workspace.<n>While discrete multi-agent finding (MAPF) methods are broadly adopted due to their scalability, their coarse discretization trajectory quality.<n>This paper tackles limitations of two approaches by introducing discrete MAPF solvers with constrained generative diffusion models.
  arXiv  Detail & Related papers  (2025-08-27T17:59:36Z)
• EvaDrive: Evolutionary Adversarial Policy Optimization for End-to-End Autonomous Driving [17.57364638932072]
  EvaDrive is a novel reinforcement learning framework for autonomous driving.<n>It provides a closed-loop adversarial framework for human-like iterative decision-making.<n>Extensive experiments on NAVSIM and Bench2Drive benchmarks demonstrate SOTA performance.
  arXiv  Detail & Related papers  (2025-08-05T11:26:28Z)
• Bench2Drive-R: Turning Real World Data into Reactive Closed-Loop Autonomous Driving Benchmark by Generative Model [63.336123527432136]
  We introduce Bench2Drive-R, a generative framework that enables reactive closed-loop evaluation.<n>Unlike existing video generative models for autonomous driving, the proposed designs are tailored for interactive simulation.<n>We compare the generation quality of Bench2Drive-R with existing generative models and achieve state-of-the-art performance.
  arXiv  Detail & Related papers  (2024-12-11T06:35:18Z)
• DiFSD: Ego-Centric Fully Sparse Paradigm with Uncertainty Denoising and Iterative Refinement for Efficient End-to-End Self-Driving [55.53171248839489]
  We propose an ego-centric fully sparse paradigm, named DiFSD, for end-to-end self-driving.<n>Specifically, DiFSD mainly consists of sparse perception, hierarchical interaction and iterative motion planner.<n>Experiments conducted on nuScenes and Bench2Drive datasets demonstrate the superior planning performance and great efficiency of DiFSD.
  arXiv  Detail & Related papers  (2024-09-15T15:55:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.