A novel framework for adaptive stress testing of autonomous vehicles in highways

• URL: http://arxiv.org/abs/2402.11813v1
• Date: Mon, 19 Feb 2024 04:02:40 GMT
• Title: A novel framework for adaptive stress testing of autonomous vehicles in highways
• Authors: Linh Trinh, Quang-Hung Luu, Thai M. Nguyen, Hai L. Vu
• Abstract summary: We propose a novel framework to explore corner cases that can result in safety concerns in a highway traffic scenario. We develop a new reward function for DRL to guide the AST in identifying crash scenarios based on the collision probability estimate. The proposed framework is further integrated with a new driving model enabling us to create more realistic traffic scenarios.
• Score: 3.2112502548606825
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Guaranteeing the safe operations of autonomous vehicles (AVs) is crucial for their widespread adoption and public acceptance. It is thus of a great significance to not only assess the AV against the standard safety tests, but also discover potential corner cases of the AV under test that could lead to unsafe behaviour or scenario. In this paper, we propose a novel framework to systematically explore corner cases that can result in safety concerns in a highway traffic scenario. The framework is based on an adaptive stress testing (AST) approach, an emerging validation method that leverages a Markov decision process to formulate the scenarios and deep reinforcement learning (DRL) to discover the desirable patterns representing corner cases. To this end, we develop a new reward function for DRL to guide the AST in identifying crash scenarios based on the collision probability estimate between the AV under test (i.e., the ego vehicle) and the trajectory of other vehicles on the highway. The proposed framework is further integrated with a new driving model enabling us to create more realistic traffic scenarios capturing both the longitudinal and lateral movements of vehicles on the highway. In our experiment, we calibrate our model using real-world crash statistics involving automated vehicles in California, and then we analyze the characteristics of the AV and the framework. Quantitative and qualitative analyses of our experimental results demonstrate that our framework outperforms other existing AST schemes. The study can help discover crash scenarios of AV that are unknown or absent in human driving, thereby enhancing the safety and trustworthiness of AV technology.

Related papers

• CRASH: Challenging Reinforcement-Learning Based Adversarial Scenarios For Safety Hardening [16.305837225117607]
  This paper introduces CRASH - Challenging Reinforcement-learning based Adversarial scenarios for Safety Hardening. First CRASH can control adversarial Non Player Character (NPC) agents in an AV simulator to automatically induce collisions with the Ego vehicle. We also propose a novel approach, that we term safety hardening, which iteratively refines the motion planner by simulating improvement scenarios against adversarial agents.
  arXiv  Detail & Related papers  (2024-11-26T00:00:27Z)
• FREA: Feasibility-Guided Generation of Safety-Critical Scenarios with Reasonable Adversariality [13.240598841087841]
  We introduce FREA, a novel safety-critical scenarios generation method that incorporates the Largest Feasible Region (LFR) of AV as guidance. Experiments illustrate that FREA can effectively generate safety-critical scenarios, yielding considerable near-miss events.
  arXiv  Detail & Related papers  (2024-06-05T06:26:15Z)
• Work-in-Progress: Crash Course: Can (Under Attack) Autonomous Driving Beat Human Drivers? [60.51287814584477]
  This paper evaluates the inherent risks in autonomous driving by examining the current landscape of AVs. We develop specific claims highlighting the delicate balance between the advantages of AVs and potential security challenges in real-world scenarios.
  arXiv  Detail & Related papers  (2024-05-14T09:42:21Z)
• PAFOT: A Position-Based Approach for Finding Optimal Tests of Autonomous Vehicles [4.243926243206826]
  This paper proposes PAFOT, a position-based approach testing framework. PAFOT generates adversarial driving scenarios to expose safety violations of Automated Driving Systems. Experiments show PAFOT can effectively generate safety-critical scenarios to crash ADSs and is able to find collisions in a short simulation time.
  arXiv  Detail & Related papers  (2024-05-06T10:04:40Z)
• CAT: Closed-loop Adversarial Training for Safe End-to-End Driving [54.60865656161679]
  Adversarial Training (CAT) is a framework for safe end-to-end driving in autonomous vehicles. Cat aims to continuously improve the safety of driving agents by training the agent on safety-critical scenarios. Cat can effectively generate adversarial scenarios countering the agent being trained.
  arXiv  Detail & Related papers  (2023-10-19T02:49:31Z)
• A Counterfactual Safety Margin Perspective on the Scoring of Autonomous Vehicles' Riskiness [52.27309191283943]
  This paper presents a data-driven framework for assessing the risk of different AVs' behaviors. We propose the notion of counterfactual safety margin, which represents the minimum deviation from nominal behavior that could cause a collision.
  arXiv  Detail & Related papers  (2023-08-02T09:48:08Z)
• Evaluating Model-free Reinforcement Learning toward Safety-critical Tasks [70.76757529955577]
  This paper revisits prior work in this scope from the perspective of state-wise safe RL. We propose Unrolling Safety Layer (USL), a joint method that combines safety optimization and safety projection. To facilitate further research in this area, we reproduce related algorithms in a unified pipeline and incorporate them into SafeRL-Kit.
  arXiv  Detail & Related papers  (2022-12-12T06:30:17Z)
• Safety Analysis of Autonomous Driving Systems Based on Model Learning [16.38592243376647]
  We present a practical verification method for safety analysis of the autonomous driving system (ADS) The main idea is to build a surrogate model that quantitatively depicts the behaviour of an ADS in the specified traffic scenario. We demonstrate the utility of the proposed approach by evaluating safety properties on the state-of-the-art ADS in literature.
  arXiv  Detail & Related papers  (2022-11-23T06:52:40Z)
• Intersection focused Situation Coverage-based Verification and Validation Framework for Autonomous Vehicles Implemented in CARLA [1.1602089225841632]
  We present a situation coverage-based (SitCov) AV-testing framework for the verification and validation (V&V) and safety assurance of AVs. SitCov AV-testing framework focuses on vehicle-to-vehicle interaction on a road intersection under different environmental and intersection configuration situations. Our code is publicly available online, anyone can use our SitCov AV-testing framework and use it or build further on top of it.
  arXiv  Detail & Related papers  (2021-12-24T02:56:56Z)
• Generating and Characterizing Scenarios for Safety Testing of Autonomous Vehicles [86.9067793493874]
  We propose efficient mechanisms to characterize and generate testing scenarios using a state-of-the-art driving simulator. We use our method to characterize real driving data from the Next Generation Simulation (NGSIM) project. We rank the scenarios by defining metrics based on the complexity of avoiding accidents and provide insights into how the AV could have minimized the probability of incurring an accident.
  arXiv  Detail & Related papers  (2021-03-12T17:00:23Z)
• Can Autonomous Vehicles Identify, Recover From, and Adapt to Distribution Shifts? [104.04999499189402]
  Out-of-training-distribution (OOD) scenarios are a common challenge of learning agents at deployment. We propose an uncertainty-aware planning method, called emphrobust imitative planning (RIP) Our method can detect and recover from some distribution shifts, reducing the overconfident and catastrophic extrapolations in OOD scenes. We introduce an autonomous car novel-scene benchmark, textttCARNOVEL, to evaluate the robustness of driving agents to a suite of tasks with distribution shifts.
  arXiv  Detail & Related papers  (2020-06-26T11:07:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.