(Re)$^2$H2O: Autonomous Driving Scenario Generation via Reversely
Regularized Hybrid Offline-and-Online Reinforcement Learning
- URL: http://arxiv.org/abs/2302.13726v2
- Date: Sat, 10 Jun 2023 09:27:51 GMT
- Title: (Re)$^2$H2O: Autonomous Driving Scenario Generation via Reversely
Regularized Hybrid Offline-and-Online Reinforcement Learning
- Authors: Haoyi Niu, Kun Ren, Yizhou Xu, Ziyuan Yang, Yichen Lin, Yi Zhang,
Jianming Hu
- Abstract summary: We learn to generate scenarios from both offline real-world and online simulation data simultaneously.
Our solution proves to produce more risky scenarios than competitive baselines.
- Score: 4.340710644468283
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Autonomous driving and its widespread adoption have long held tremendous
promise. Nevertheless, without a trustworthy and thorough testing procedure,
not only does the industry struggle to mass-produce autonomous vehicles (AV),
but neither the general public nor policymakers are convinced to accept the
innovations. Generating safety-critical scenarios that present significant
challenges to AV is an essential first step in testing. Real-world datasets
include naturalistic but overly safe driving behaviors, whereas simulation
would allow for unrestricted exploration of diverse and aggressive traffic
scenarios. Conversely, higher-dimensional searching space in simulation
disables efficient scenario generation without real-world data distribution as
implicit constraints. In order to marry the benefits of both, it seems
appealing to learn to generate scenarios from both offline real-world and
online simulation data simultaneously. Therefore, we tailor a Reversely
Regularized Hybrid Offline-and-Online ((Re)$^2$H2O) Reinforcement Learning
recipe to additionally penalize Q-values on real-world data and reward Q-values
on simulated data, which ensures the generated scenarios are both varied and
adversarial. Through extensive experiments, our solution proves to produce more
risky scenarios than competitive baselines and it can generalize to work with
various autonomous driving models. In addition, these generated scenarios are
also corroborated to be capable of fine-tuning AV performance.
Related papers
- SAFE-SIM: Safety-Critical Closed-Loop Traffic Simulation with Controllable Adversaries [94.84458417662407]
We introduce SAFE-SIM, a novel diffusion-based controllable closed-loop safety-critical simulation framework.
We develop a novel approach to simulate safety-critical scenarios through an adversarial term in the denoising process.
We validate our framework empirically using the NuScenes dataset, demonstrating improvements in both realism and controllability.
arXiv Detail & Related papers (2023-12-31T04:14:43Z) - RealGen: Retrieval Augmented Generation for Controllable Traffic
Scenarios [62.89459646611976]
RealGen is a novel retrieval-based in-context learning framework for traffic scenario generation.
RealGen synthesizes new scenarios by combining behaviors from multiple retrieved examples in a gradient-free way.
This in-context learning framework endows versatile generative capabilities, including the ability to edit scenarios.
arXiv Detail & Related papers (2023-12-19T23:11:06Z) - ReMAV: Reward Modeling of Autonomous Vehicles for Finding Likely Failure
Events [1.84926694477846]
We propose a black-box testing framework that uses offline trajectories first to analyze the existing behavior of autonomous vehicles.
Our experiment shows an increase in 35, 23, 48, and 50% in the occurrences of vehicle collision, road object collision, pedestrian collision, and offroad steering events.
arXiv Detail & Related papers (2023-08-28T13:09:00Z) - Generative AI-empowered Simulation for Autonomous Driving in Vehicular
Mixed Reality Metaverses [130.15554653948897]
In vehicular mixed reality (MR) Metaverse, distance between physical and virtual entities can be overcome.
Large-scale traffic and driving simulation via realistic data collection and fusion from the physical world is difficult and costly.
We propose an autonomous driving architecture, where generative AI is leveraged to synthesize unlimited conditioned traffic and driving data in simulations.
arXiv Detail & Related papers (2023-02-16T16:54:10Z) - DR2L: Surfacing Corner Cases to Robustify Autonomous Driving via Domain
Randomization Reinforcement Learning [4.040937987024427]
Domain Randomization(DR) is a methodology that can bridge this gap with little or no real-world data.
An adversarial model is put forward to robustify DeepRL-based autonomous vehicles trained in simulation.
arXiv Detail & Related papers (2021-07-25T09:15:46Z) - Generating and Characterizing Scenarios for Safety Testing of Autonomous
Vehicles [86.9067793493874]
We propose efficient mechanisms to characterize and generate testing scenarios using a state-of-the-art driving simulator.
We use our method to characterize real driving data from the Next Generation Simulation (NGSIM) project.
We rank the scenarios by defining metrics based on the complexity of avoiding accidents and provide insights into how the AV could have minimized the probability of incurring an accident.
arXiv Detail & Related papers (2021-03-12T17:00:23Z) - Deep Structured Reactive Planning [94.92994828905984]
We propose a novel data-driven, reactive planning objective for self-driving vehicles.
We show that our model outperforms a non-reactive variant in successfully completing highly complex maneuvers.
arXiv Detail & Related papers (2021-01-18T01:43:36Z) - TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviors [74.67698916175614]
We propose TrafficSim, a multi-agent behavior model for realistic traffic simulation.
In particular, we leverage an implicit latent variable model to parameterize a joint actor policy.
We show TrafficSim generates significantly more realistic and diverse traffic scenarios as compared to a diverse set of baselines.
arXiv Detail & Related papers (2021-01-17T00:29:30Z) - Towards Automated Safety Coverage and Testing for Autonomous Vehicles
with Reinforcement Learning [0.3683202928838613]
Validation puts the autonomous vehicle system to the test in scenarios or situations that the system would likely encounter in everyday driving.
We propose using reinforcement learning (RL) to generate failure examples and unexpected traffic situations for the AV software implementation.
arXiv Detail & Related papers (2020-05-22T19:00:38Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.