Adversarial Evaluation of Autonomous Vehicles in Lane-Change Scenarios
- URL: http://arxiv.org/abs/2004.06531v2
- Date: Mon, 23 Nov 2020 20:27:40 GMT
- Title: Adversarial Evaluation of Autonomous Vehicles in Lane-Change Scenarios
- Authors: Baiming Chen, Xiang Chen, Wu Qiong, Liang Li
- Abstract summary: We propose an adaptive evaluation framework to efficiently evaluate autonomous vehicles in adversarial environments.
Considering the multimodal nature of dangerous scenarios, we use ensemble models to represent different local optimums for diversity.
Results show that the adversarial scenarios generated by our method significantly degrade the performance of the tested vehicles.
- Score: 10.53961877853783
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Autonomous vehicles must be comprehensively evaluated before deployed in
cities and highways. However, most existing evaluation approaches for
autonomous vehicles are static and lack adaptability, so they are usually
inefficient in generating challenging scenarios for tested vehicles. In this
paper, we propose an adaptive evaluation framework to efficiently evaluate
autonomous vehicles in adversarial environments generated by deep reinforcement
learning. Considering the multimodal nature of dangerous scenarios, we use
ensemble models to represent different local optimums for diversity. We then
utilize a nonparametric Bayesian method to cluster the adversarial policies.
The proposed method is validated in a typical lane-change scenario that
involves frequent interactions between the ego vehicle and the surrounding
vehicles. Results show that the adversarial scenarios generated by our method
significantly degrade the performance of the tested vehicles. We also
illustrate different patterns of generated adversarial environments, which can
be used to infer the weaknesses of the tested vehicles.
Related papers
- Work-in-Progress: Crash Course: Can (Under Attack) Autonomous Driving Beat Human Drivers? [60.51287814584477]
This paper evaluates the inherent risks in autonomous driving by examining the current landscape of AVs.
We develop specific claims highlighting the delicate balance between the advantages of AVs and potential security challenges in real-world scenarios.
arXiv Detail & Related papers (2024-05-14T09:42:21Z) - SAFE-SIM: Safety-Critical Closed-Loop Traffic Simulation with Controllable Adversaries [94.84458417662407]
We introduce SAFE-SIM, a novel diffusion-based controllable closed-loop safety-critical simulation framework.
We develop a novel approach to simulate safety-critical scenarios through an adversarial term in the denoising process.
We validate our framework empirically using the NuScenes dataset, demonstrating improvements in both realism and controllability.
arXiv Detail & Related papers (2023-12-31T04:14:43Z) - Parameterized Decision-making with Multi-modal Perception for Autonomous
Driving [12.21578713219778]
We propose a parameterized decision-making framework with multi-modal perception based on deep reinforcement learning, called AUTO.
A hybrid reward function takes into account aspects of safety, traffic efficiency, passenger comfort, and impact to guide the framework to generate optimal actions.
arXiv Detail & Related papers (2023-12-19T08:27:02Z) - Interaction-Aware Decision-Making for Autonomous Vehicles in Forced
Merging Scenario Leveraging Social Psychology Factors [7.812717451846781]
We consider a behavioral model that incorporates both social behaviors and personal objectives of the interacting drivers.
We develop a receding-horizon control-based decision-making strategy that estimates online the other drivers' intentions.
arXiv Detail & Related papers (2023-09-25T19:49:14Z) - Autonomous and Human-Driven Vehicles Interacting in a Roundabout: A
Quantitative and Qualitative Evaluation [34.67306374722473]
We learn a policy to minimize traffic jams and to minimize pollution in a roundabout in Milan, Italy.
We qualitatively evaluate the learned policy using a cutting-edge cockpit to assess its performance in near-real-world conditions.
Our findings show that human-driven vehicles benefit from optimizing AVs dynamics.
arXiv Detail & Related papers (2023-09-15T09:02:16Z) - ReMAV: Reward Modeling of Autonomous Vehicles for Finding Likely Failure
Events [1.84926694477846]
We propose a black-box testing framework that uses offline trajectories first to analyze the existing behavior of autonomous vehicles.
Our experiment shows an increase in 35, 23, 48, and 50% in the occurrences of vehicle collision, road object collision, pedestrian collision, and offroad steering events.
arXiv Detail & Related papers (2023-08-28T13:09:00Z) - Evaluating the Robustness of Deep Reinforcement Learning for Autonomous
Policies in a Multi-agent Urban Driving Environment [3.8073142980733]
We propose a benchmarking framework for the comparison of deep reinforcement learning in a vision-based autonomous driving.
We run the experiments in a vision-only high-fidelity urban driving simulated environments.
The results indicate that only some of the deep reinforcement learning algorithms perform consistently better across single and multi-agent scenarios.
arXiv Detail & Related papers (2021-12-22T15:14:50Z) - Generating and Characterizing Scenarios for Safety Testing of Autonomous
Vehicles [86.9067793493874]
We propose efficient mechanisms to characterize and generate testing scenarios using a state-of-the-art driving simulator.
We use our method to characterize real driving data from the Next Generation Simulation (NGSIM) project.
We rank the scenarios by defining metrics based on the complexity of avoiding accidents and provide insights into how the AV could have minimized the probability of incurring an accident.
arXiv Detail & Related papers (2021-03-12T17:00:23Z) - LookOut: Diverse Multi-Future Prediction and Planning for Self-Driving [139.33800431159446]
LookOut is an approach to jointly perceive the environment and predict a diverse set of futures from sensor data.
We show that our model demonstrates significantly more diverse and sample-efficient motion forecasting in a large-scale self-driving dataset.
arXiv Detail & Related papers (2021-01-16T23:19:22Z) - Studying Person-Specific Pointing and Gaze Behavior for Multimodal
Referencing of Outside Objects from a Moving Vehicle [58.720142291102135]
Hand pointing and eye gaze have been extensively investigated in automotive applications for object selection and referencing.
Existing outside-the-vehicle referencing methods focus on a static situation, whereas the situation in a moving vehicle is highly dynamic and subject to safety-critical constraints.
We investigate the specific characteristics of each modality and the interaction between them when used in the task of referencing outside objects.
arXiv Detail & Related papers (2020-09-23T14:56:19Z) - Can Autonomous Vehicles Identify, Recover From, and Adapt to
Distribution Shifts? [104.04999499189402]
Out-of-training-distribution (OOD) scenarios are a common challenge of learning agents at deployment.
We propose an uncertainty-aware planning method, called emphrobust imitative planning (RIP)
Our method can detect and recover from some distribution shifts, reducing the overconfident and catastrophic extrapolations in OOD scenes.
We introduce an autonomous car novel-scene benchmark, textttCARNOVEL, to evaluate the robustness of driving agents to a suite of tasks with distribution shifts.
arXiv Detail & Related papers (2020-06-26T11:07:32Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.