Adversarial Evaluation of Autonomous Vehicles in Lane-Change Scenarios

• URL: http://arxiv.org/abs/2004.06531v2
• Date: Mon, 23 Nov 2020 20:27:40 GMT
• Title: Adversarial Evaluation of Autonomous Vehicles in Lane-Change Scenarios
• Authors: Baiming Chen, Xiang Chen, Wu Qiong, Liang Li
• Abstract summary: We propose an adaptive evaluation framework to efficiently evaluate autonomous vehicles in adversarial environments. Considering the multimodal nature of dangerous scenarios, we use ensemble models to represent different local optimums for diversity. Results show that the adversarial scenarios generated by our method significantly degrade the performance of the tested vehicles.
• Score: 10.53961877853783
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Autonomous vehicles must be comprehensively evaluated before deployed in cities and highways. However, most existing evaluation approaches for autonomous vehicles are static and lack adaptability, so they are usually inefficient in generating challenging scenarios for tested vehicles. In this paper, we propose an adaptive evaluation framework to efficiently evaluate autonomous vehicles in adversarial environments generated by deep reinforcement learning. Considering the multimodal nature of dangerous scenarios, we use ensemble models to represent different local optimums for diversity. We then utilize a nonparametric Bayesian method to cluster the adversarial policies. The proposed method is validated in a typical lane-change scenario that involves frequent interactions between the ego vehicle and the surrounding vehicles. Results show that the adversarial scenarios generated by our method significantly degrade the performance of the tested vehicles. We also illustrate different patterns of generated adversarial environments, which can be used to infer the weaknesses of the tested vehicles.

Related papers

• Generating Out-Of-Distribution Scenarios Using Language Models [58.47597351184034]
  Large Language Models (LLMs) have shown promise in autonomous driving. This paper introduces a framework for generating diverse Out-Of-Distribution (OOD) driving scenarios. We evaluate our framework through extensive simulations and introduce a new "OOD-ness" metric.
  arXiv  Detail & Related papers  (2024-11-25T16:38:17Z)
• Foundation Models for Rapid Autonomy Validation [4.417336418010182]
  A key challenge is that an autonomous vehicle requires testing in every kind of driving scenario it could encounter. We propose the use of a behavior foundation model, specifically a masked autoencoder (MAE), trained to reconstruct driving scenarios.
  arXiv  Detail & Related papers  (2024-10-22T15:32:43Z)
• Bench4Merge: A Comprehensive Benchmark for Merging in Realistic Dense Traffic with Micro-Interactive Vehicles [20.832829903505296]
  We develop a benchmark for assessing motion planning capabilities in merging scenarios. Our approach involves other vehicles trained in large scale datasets with micro-behavioral characteristics. Extensive experiments have demonstrated the advanced nature of this evaluation benchmark.
  arXiv  Detail & Related papers  (2024-10-21T11:35:33Z)
• AdvDiffuser: Generating Adversarial Safety-Critical Driving Scenarios via Guided Diffusion [6.909801263560482]
  AdvDiffuser is an adversarial framework for generating safety-critical driving scenarios through guided diffusion. We show that AdvDiffuser can be applied to various tested systems with minimal warm-up episode data.
  arXiv  Detail & Related papers  (2024-10-11T02:03:21Z)
• Work-in-Progress: Crash Course: Can (Under Attack) Autonomous Driving Beat Human Drivers? [60.51287814584477]
  This paper evaluates the inherent risks in autonomous driving by examining the current landscape of AVs. We develop specific claims highlighting the delicate balance between the advantages of AVs and potential security challenges in real-world scenarios.
  arXiv  Detail & Related papers  (2024-05-14T09:42:21Z)
• SAFE-SIM: Safety-Critical Closed-Loop Traffic Simulation with Diffusion-Controllable Adversaries [94.84458417662407]
  We introduce SAFE-SIM, a controllable closed-loop safety-critical simulation framework. Our approach yields two distinct advantages: 1) generating realistic long-tail safety-critical scenarios that closely reflect real-world conditions, and 2) providing controllable adversarial behavior for more comprehensive and interactive evaluations. We validate our framework empirically using the nuScenes and nuPlan datasets across multiple planners, demonstrating improvements in both realism and controllability.
  arXiv  Detail & Related papers  (2023-12-31T04:14:43Z)
• Autonomous and Human-Driven Vehicles Interacting in a Roundabout: A Quantitative and Qualitative Evaluation [34.67306374722473]
  We learn a policy to minimize traffic jams and to minimize pollution in a roundabout in Milan, Italy. We qualitatively evaluate the learned policy using a cutting-edge cockpit to assess its performance in near-real-world conditions. Our findings show that human-driven vehicles benefit from optimizing AVs dynamics.
  arXiv  Detail & Related papers  (2023-09-15T09:02:16Z)
• ReMAV: Reward Modeling of Autonomous Vehicles for Finding Likely Failure Events [1.84926694477846]
  We propose a black-box testing framework that uses offline trajectories first to analyze the existing behavior of autonomous vehicles. Our experiment shows an increase in 35, 23, 48, and 50% in the occurrences of vehicle collision, road object collision, pedestrian collision, and offroad steering events.
  arXiv  Detail & Related papers  (2023-08-28T13:09:00Z)
• LookOut: Diverse Multi-Future Prediction and Planning for Self-Driving [139.33800431159446]
  LookOut is an approach to jointly perceive the environment and predict a diverse set of futures from sensor data. We show that our model demonstrates significantly more diverse and sample-efficient motion forecasting in a large-scale self-driving dataset.
  arXiv  Detail & Related papers  (2021-01-16T23:19:22Z)
• Studying Person-Specific Pointing and Gaze Behavior for Multimodal Referencing of Outside Objects from a Moving Vehicle [58.720142291102135]
  Hand pointing and eye gaze have been extensively investigated in automotive applications for object selection and referencing. Existing outside-the-vehicle referencing methods focus on a static situation, whereas the situation in a moving vehicle is highly dynamic and subject to safety-critical constraints. We investigate the specific characteristics of each modality and the interaction between them when used in the task of referencing outside objects.
  arXiv  Detail & Related papers  (2020-09-23T14:56:19Z)
• Can Autonomous Vehicles Identify, Recover From, and Adapt to Distribution Shifts? [104.04999499189402]
  Out-of-training-distribution (OOD) scenarios are a common challenge of learning agents at deployment. We propose an uncertainty-aware planning method, called emphrobust imitative planning (RIP) Our method can detect and recover from some distribution shifts, reducing the overconfident and catastrophic extrapolations in OOD scenes. We introduce an autonomous car novel-scene benchmark, textttCARNOVEL, to evaluate the robustness of driving agents to a suite of tasks with distribution shifts.
  arXiv  Detail & Related papers  (2020-06-26T11:07:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.