ReconDreamer-RL: Enhancing Reinforcement Learning via Diffusion-based Scene Reconstruction

• URL: http://arxiv.org/abs/2508.08170v2
• Date: Thu, 21 Aug 2025 11:45:55 GMT
• Title: ReconDreamer-RL: Enhancing Reinforcement Learning via Diffusion-based Scene Reconstruction
• Authors: Chaojun Ni, Guosheng Zhao, Xiaofeng Wang, Zheng Zhu, Wenkang Qin, Xinze Chen, Guanghong Jia, Guan Huang, Wenjun Mei,
• Abstract summary: ReconDreamer-RL is a framework designed to integrate video diffusion priors into scene reconstruction to aid reinforcement learning.<n>We show that ReconDreamer-RL improves end-to-end autonomous driving training, outperforming imitation learning methods with a 5x reduction in the Collision Ratio.
• Score: 26.402373173809753
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Reinforcement learning for training end-to-end autonomous driving models in closed-loop simulations is gaining growing attention. However, most simulation environments differ significantly from real-world conditions, creating a substantial simulation-to-reality (sim2real) gap. To bridge this gap, some approaches utilize scene reconstruction techniques to create photorealistic environments as a simulator. While this improves realistic sensor simulation, these methods are inherently constrained by the distribution of the training data, making it difficult to render high-quality sensor data for novel trajectories or corner case scenarios. Therefore, we propose ReconDreamer-RL, a framework designed to integrate video diffusion priors into scene reconstruction to aid reinforcement learning, thereby enhancing end-to-end autonomous driving training. Specifically, in ReconDreamer-RL, we introduce ReconSimulator, which combines the video diffusion prior for appearance modeling and incorporates a kinematic model for physical modeling, thereby reconstructing driving scenarios from real-world data. This narrows the sim2real gap for closed-loop evaluation and reinforcement learning. To cover more corner-case scenarios, we introduce the Dynamic Adversary Agent (DAA), which adjusts the trajectories of surrounding vehicles relative to the ego vehicle, autonomously generating corner-case traffic scenarios (e.g., cut-in). Finally, the Cousin Trajectory Generator (CTG) is proposed to address the issue of training data distribution, which is often biased toward simple straight-line movements. Experiments show that ReconDreamer-RL improves end-to-end autonomous driving training, outperforming imitation learning methods with a 5x reduction in the Collision Ratio.

Related papers

• DiffusionHarmonizer: Bridging Neural Reconstruction and Photorealistic Simulation with Online Diffusion Enhancer [62.18680935878919]
  We introduce DiffusionHarmonizer, an online generative enhancement framework that transforms renderings into temporally consistent outputs.<n>At its core is a single-step temporally-conditioned enhancer capable of running in online simulators on a single GPU.
  arXiv  Detail & Related papers  (2026-02-27T15:35:30Z)
• Dynamics-Decoupled Trajectory Alignment for Sim-to-Real Transfer in Reinforcement Learning for Autonomous Driving [37.61391230495979]
  Reinforcement learning (RL) has shown promise in robotics, but deploying RL on real vehicles remains challenging.<n> Factors such as tire characteristics, road surface conditions, aerodynamic disturbances, and vehicle load make it infeasible to model real-world dynamics accurately.<n>We present a framework that decouples motion planning from vehicle control through a spatial and temporal alignment strategy.
  arXiv  Detail & Related papers  (2025-11-10T14:45:24Z)
• Scenario Dreamer: Vectorized Latent Diffusion for Generating Driving Simulation Environments [39.649242431308544]
  We introduce Scenario Dreamer, a fully data-driven generative simulator for autonomous vehicle planning.<n>It generates both the initial traffic scene - comprising a lane graph and agent bounding boxes - and closed-loop agent behaviours.<n>Case study shows that Scenario Dreamer achieves superior generation quality with around 2x fewer parameters, 6x lower generation, and 10x training hours compared to the strongest baseline.
  arXiv  Detail & Related papers  (2025-03-28T15:03:41Z)
• Vid2Sim: Realistic and Interactive Simulation from Video for Urban Navigation [62.5805866419814]
  Vid2Sim is a novel framework that bridges the sim2real gap through a scalable and cost-efficient real2sim pipeline for neural 3D scene reconstruction and simulation.<n>Experiments demonstrate that Vid2Sim significantly improves the performance of urban navigation in the digital twins and real world by 31.2% and 68.3% in success rate.
  arXiv  Detail & Related papers  (2025-01-12T03:01:15Z)
• Sim-to-Real Transfer of Deep Reinforcement Learning Agents for Online Coverage Path Planning [15.792914346054502]
  We tackle the challenge of sim-to-real transfer of reinforcement learning (RL) agents for coverage path planning ( CPP) We bridge the sim-to-real gap through a semi-virtual environment, including a real robot and real-time aspects, while utilizing a simulated sensor and obstacles. We find that a high inference frequency allows first-order Markovian policies to transfer directly from simulation, while higher-order policies can be fine-tuned to further reduce the sim-to-real gap.
  arXiv  Detail & Related papers  (2024-06-07T13:24:19Z)
• S-NeRF++: Autonomous Driving Simulation via Neural Reconstruction and Generation [21.501865765631123]
  S-NeRF++ is an innovative autonomous driving simulation system based on neural reconstruction.<n>S-NeRF++ is trained on widely-used self-driving datasets such as nuScenes and radiance.<n>System effectively utilizes noisy and sparse LiDAR data to refine training and address depth outliers.
  arXiv  Detail & Related papers  (2024-02-03T10:35:42Z)
• Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Research [76.93956925360638]
  Waymax is a new data-driven simulator for autonomous driving in multi-agent scenes. It runs entirely on hardware accelerators such as TPUs/GPUs and supports in-graph simulation for training. We benchmark a suite of popular imitation and reinforcement learning algorithms with ablation studies on different design decisions.
  arXiv  Detail & Related papers  (2023-10-12T20:49:15Z)
• Continual learning autoencoder training for a particle-in-cell simulation via streaming [52.77024349608834]
  upcoming exascale era will provide a new generation of physics simulations with high resolution. These simulations will have a high resolution, which will impact the training of machine learning models since storing a high amount of simulation data on disk is nearly impossible. This work presents an approach that trains a neural network concurrently to a running simulation without data on a disk.
  arXiv  Detail & Related papers  (2022-11-09T09:55:14Z)
• Sim-to-Real via Sim-to-Seg: End-to-end Off-road Autonomous Driving Without Real Data [56.49494318285391]
  We present Sim2Seg, a re-imagining of RCAN that crosses the visual reality gap for off-road autonomous driving. This is done by learning to translate randomized simulation images into simulated segmentation and depth maps. This allows us to train an end-to-end RL policy in simulation, and directly deploy in the real-world.
  arXiv  Detail & Related papers  (2022-10-25T17:50:36Z)
• DR2L: Surfacing Corner Cases to Robustify Autonomous Driving via Domain Randomization Reinforcement Learning [4.040937987024427]
  Domain Randomization(DR) is a methodology that can bridge this gap with little or no real-world data. An adversarial model is put forward to robustify DeepRL-based autonomous vehicles trained in simulation.
  arXiv  Detail & Related papers  (2021-07-25T09:15:46Z)
• TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviors [74.67698916175614]
  We propose TrafficSim, a multi-agent behavior model for realistic traffic simulation. In particular, we leverage an implicit latent variable model to parameterize a joint actor policy. We show TrafficSim generates significantly more realistic and diverse traffic scenarios as compared to a diverse set of baselines.
  arXiv  Detail & Related papers  (2021-01-17T00:29:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.