Continual Driving Policy Optimization with Closed-Loop Individualized
Curricula
- URL: http://arxiv.org/abs/2309.14209v3
- Date: Wed, 6 Mar 2024 07:55:17 GMT
- Title: Continual Driving Policy Optimization with Closed-Loop Individualized
Curricula
- Authors: Haoyi Niu, Yizhou Xu, Xingjian Jiang, Jianming Hu
- Abstract summary: We develop a continuous driving policy optimization framework featuring Closed-Loop Individualized Curricula (CLIC)
CLIC frames AV Evaluation as a collision prediction task, where it estimates the chance of AV failures in these scenarios at each iteration.
We show that CLIC surpasses other curriculum-based training strategies, showing substantial improvement in managing risky scenarios.
- Score: 3.171483862183451
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The safety of autonomous vehicles (AV) has been a long-standing top concern,
stemming from the absence of rare and safety-critical scenarios in the
long-tail naturalistic driving distribution. To tackle this challenge, a surge
of research in scenario-based autonomous driving has emerged, with a focus on
generating high-risk driving scenarios and applying them to conduct
safety-critical testing of AV models. However, limited work has been explored
on the reuse of these extensive scenarios to iteratively improve AV models.
Moreover, it remains intractable and challenging to filter through gigantic
scenario libraries collected from other AV models with distinct behaviors,
attempting to extract transferable information for current AV improvement.
Therefore, we develop a continual driving policy optimization framework
featuring Closed-Loop Individualized Curricula (CLIC), which we factorize into
a set of standardized sub-modules for flexible implementation choices: AV
Evaluation, Scenario Selection, and AV Training. CLIC frames AV Evaluation as a
collision prediction task, where it estimates the chance of AV failures in
these scenarios at each iteration. Subsequently, by re-sampling from historical
scenarios based on these failure probabilities, CLIC tailors individualized
curricula for downstream training, aligning them with the evaluated capability
of AV. Accordingly, CLIC not only maximizes the utilization of the vast
pre-collected scenario library for closed-loop driving policy optimization but
also facilitates AV improvement by individualizing its training with more
challenging cases out of those poorly organized scenarios. Experimental results
clearly indicate that CLIC surpasses other curriculum-based training
strategies, showing substantial improvement in managing risky scenarios, while
still maintaining proficiency in handling simpler cases.
Related papers
- FREA: Feasibility-Guided Generation of Safety-Critical Scenarios with Reasonable Adversariality [13.240598841087841]
We introduce FREA, a novel safety-critical scenarios generation method.
It incorporates the Largest Feasible Region (LFR) of AV as guidance to ensure the reasonableness of the adversarial scenarios.
Experiments show that FREA can effectively generate safety-critical scenarios, yielding considerable near-miss events.
arXiv Detail & Related papers (2024-06-05T06:26:15Z) - Enhancing Autonomous Vehicle Training with Language Model Integration and Critical Scenario Generation [32.02261963851354]
CRITICAL is a novel closed-loop framework for autonomous vehicle (AV) training and testing.
The framework achieves this by integrating real-world traffic dynamics, driving behavior analysis, surrogate safety measures, and an optional Large Language Model (LLM) component.
arXiv Detail & Related papers (2024-04-12T16:13:10Z) - A novel framework for adaptive stress testing of autonomous vehicles in
highways [3.2112502548606825]
We propose a novel framework to explore corner cases that can result in safety concerns in a highway traffic scenario.
We develop a new reward function for DRL to guide the AST in identifying crash scenarios based on the collision probability estimate.
The proposed framework is further integrated with a new driving model enabling us to create more realistic traffic scenarios.
arXiv Detail & Related papers (2024-02-19T04:02:40Z) - SAFE-SIM: Safety-Critical Closed-Loop Traffic Simulation with Controllable Adversaries [94.84458417662407]
We introduce SAFE-SIM, a novel diffusion-based controllable closed-loop safety-critical simulation framework.
We develop a novel approach to simulate safety-critical scenarios through an adversarial term in the denoising process.
We validate our framework empirically using the NuScenes dataset, demonstrating improvements in both realism and controllability.
arXiv Detail & Related papers (2023-12-31T04:14:43Z) - CAT: Closed-loop Adversarial Training for Safe End-to-End Driving [54.60865656161679]
Adversarial Training (CAT) is a framework for safe end-to-end driving in autonomous vehicles.
Cat aims to continuously improve the safety of driving agents by training the agent on safety-critical scenarios.
Cat can effectively generate adversarial scenarios countering the agent being trained.
arXiv Detail & Related papers (2023-10-19T02:49:31Z) - Stackelberg Driver Model for Continual Policy Improvement in
Scenario-Based Closed-Loop Autonomous Driving [5.765939495779461]
adversarial generation methods have emerged as a class of efficient approaches to synthesize safety-critical scenarios.
We tailor the Stackelberg Driver Model (SDM) to accurately characterize the hierarchical nature of vehicle interaction dynamics.
Our algorithm exhibits superior performance compared to several baselines especially in higher dimensional scenarios.
arXiv Detail & Related papers (2023-09-25T15:47:07Z) - A Counterfactual Safety Margin Perspective on the Scoring of Autonomous
Vehicles' Riskiness [52.27309191283943]
This paper presents a data-driven framework for assessing the risk of different AVs' behaviors.
We propose the notion of counterfactual safety margin, which represents the minimum deviation from nominal behavior that could cause a collision.
arXiv Detail & Related papers (2023-08-02T09:48:08Z) - Automatically Learning Fallback Strategies with Model-Free Reinforcement
Learning in Safety-Critical Driving Scenarios [9.761912672523977]
We present a principled approach for a model-free Reinforcement Learning (RL) agent to capture multiple modes of behaviour in an environment.
We introduce an extra pseudo-reward term to the reward model, to encourage exploration to areas of state-space different from areas privileged by the optimal policy.
We show that we are able to learn useful policies that would have otherwise been missed out on during training, and unavailable to use when executing the control algorithm.
arXiv Detail & Related papers (2022-04-11T15:34:49Z) - Generating Useful Accident-Prone Driving Scenarios via a Learned Traffic
Prior [135.78858513845233]
STRIVE is a method to automatically generate challenging scenarios that cause a given planner to produce undesirable behavior, like collisions.
To maintain scenario plausibility, the key idea is to leverage a learned model of traffic motion in the form of a graph-based conditional VAE.
A subsequent optimization is used to find a "solution" to the scenario, ensuring it is useful to improve the given planner.
arXiv Detail & Related papers (2021-12-09T18:03:27Z) - LookOut: Diverse Multi-Future Prediction and Planning for Self-Driving [139.33800431159446]
LookOut is an approach to jointly perceive the environment and predict a diverse set of futures from sensor data.
We show that our model demonstrates significantly more diverse and sample-efficient motion forecasting in a large-scale self-driving dataset.
arXiv Detail & Related papers (2021-01-16T23:19:22Z) - Can Autonomous Vehicles Identify, Recover From, and Adapt to
Distribution Shifts? [104.04999499189402]
Out-of-training-distribution (OOD) scenarios are a common challenge of learning agents at deployment.
We propose an uncertainty-aware planning method, called emphrobust imitative planning (RIP)
Our method can detect and recover from some distribution shifts, reducing the overconfident and catastrophic extrapolations in OOD scenes.
We introduce an autonomous car novel-scene benchmark, textttCARNOVEL, to evaluate the robustness of driving agents to a suite of tasks with distribution shifts.
arXiv Detail & Related papers (2020-06-26T11:07:32Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.