Realistic Safety-critical Scenarios Search for Autonomous Driving System via Behavior Tree

• URL: http://arxiv.org/abs/2305.06603v1
• Date: Thu, 11 May 2023 06:53:03 GMT
• Title: Realistic Safety-critical Scenarios Search for Autonomous Driving System via Behavior Tree
• Authors: Ping Zhang, Lingfeng Ming, Tingyi Yuan, Cong Qiu, Yang Li, Xinhua Hui, Zhiquan Zhang, Chao Huang
• Abstract summary: We propose the Matrix-Fuzzer, a behavior tree-based testing framework, to automatically generate realistic safety-critical test scenarios. Our approach is able to find the most types of safety-critical scenarios, but only generating around 30% of the total scenarios compared with the baseline algorithm.
• Score: 8.286351881735191
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The simulation-based testing of Autonomous Driving Systems (ADSs) has gained significant attention. However, current approaches often fall short of accurately assessing ADSs for two reasons: over-reliance on expert knowledge and the utilization of simplistic evaluation metrics. That leads to discrepancies between simulated scenarios and naturalistic driving environments. To address this, we propose the Matrix-Fuzzer, a behavior tree-based testing framework, to automatically generate realistic safety-critical test scenarios. Our approach involves the $log2BT$ method, which abstracts logged road-users' trajectories to behavior sequences. Furthermore, we vary the properties of behaviors from real-world driving distributions and then use an adaptive algorithm to explore the input space. Meanwhile, we design a general evaluation engine that guides the algorithm toward critical areas, thus reducing the generation of invalid scenarios. Our approach is demonstrated in our Matrix Simulator. The experimental results show that: (1) Our $log2BT$ achieves satisfactory trajectory reconstructions. (2) Our approach is able to find the most types of safety-critical scenarios, but only generating around 30% of the total scenarios compared with the baseline algorithm. Specifically, it improves the ratio of the critical violations to total scenarios and the ratio of the types to total scenarios by at least 10x and 5x, respectively, while reducing the ratio of the invalid scenarios to total scenarios by at least 58% in two case studies.

Related papers

• Foundation Models for Rapid Autonomy Validation [4.417336418010182]
  A key challenge is that an autonomous vehicle requires testing in every kind of driving scenario it could encounter. We propose the use of a behavior foundation model, specifically a masked autoencoder (MAE), trained to reconstruct driving scenarios.
  arXiv  Detail & Related papers  (2024-10-22T15:32:43Z)
• ReGentS: Real-World Safety-Critical Driving Scenario Generation Made Stable [88.08120417169971]
  Machine learning based autonomous driving systems often face challenges with safety-critical scenarios that are rare in real-world data. This work explores generating safety-critical driving scenarios by modifying complex real-world regular scenarios through trajectory optimization. Our approach addresses unrealistic diverging trajectories and unavoidable collision scenarios that are not useful for training robust planner.
  arXiv  Detail & Related papers  (2024-09-12T08:26:33Z)
• Adversarial Safety-Critical Scenario Generation using Naturalistic Human Driving Priors [2.773055342671194]
  We introduce a natural adversarial scenario generation solution using naturalistic human driving priors and reinforcement learning techniques. Our findings demonstrate that the proposed model can generate realistic safety-critical test scenarios covering both naturalness and adversariality.
  arXiv  Detail & Related papers  (2024-08-06T13:58:56Z)
• Planning with Adaptive World Models for Autonomous Driving [50.4439896514353]
  Motion planners (MPs) are crucial for safe navigation in complex urban environments. nuPlan, a recently released MP benchmark, addresses this limitation by augmenting real-world driving logs with closed-loop simulation logic. We present AdaptiveDriver, a model-predictive control (MPC) based planner that unrolls different world models conditioned on BehaviorNet's predictions.
  arXiv  Detail & Related papers  (2024-06-15T18:53:45Z)
• SAFE-SIM: Safety-Critical Closed-Loop Traffic Simulation with Diffusion-Controllable Adversaries [94.84458417662407]
  We introduce SAFE-SIM, a controllable closed-loop safety-critical simulation framework. Our approach yields two distinct advantages: 1) generating realistic long-tail safety-critical scenarios that closely reflect real-world conditions, and 2) providing controllable adversarial behavior for more comprehensive and interactive evaluations. We validate our framework empirically using the nuScenes and nuPlan datasets across multiple planners, demonstrating improvements in both realism and controllability.
  arXiv  Detail & Related papers  (2023-12-31T04:14:43Z)
• An Application of Scenario Exploration to Find New Scenarios for the Development and Testing of Automated Driving Systems in Urban Scenarios [2.480533141352916]
  This work aims to find relevant, interesting, or critical parameter sets within logical scenarios by utilizing Bayes optimization and Gaussian processes. A list of ideas this work leads to and should be investigated further is presented.
  arXiv  Detail & Related papers  (2022-05-17T09:47:32Z)
• Generating Useful Accident-Prone Driving Scenarios via a Learned Traffic Prior [135.78858513845233]
  STRIVE is a method to automatically generate challenging scenarios that cause a given planner to produce undesirable behavior, like collisions. To maintain scenario plausibility, the key idea is to leverage a learned model of traffic motion in the form of a graph-based conditional VAE. A subsequent optimization is used to find a "solution" to the scenario, ensuring it is useful to improve the given planner.
  arXiv  Detail & Related papers  (2021-12-09T18:03:27Z)
• Generating and Characterizing Scenarios for Safety Testing of Autonomous Vehicles [86.9067793493874]
  We propose efficient mechanisms to characterize and generate testing scenarios using a state-of-the-art driving simulator. We use our method to characterize real driving data from the Next Generation Simulation (NGSIM) project. We rank the scenarios by defining metrics based on the complexity of avoiding accidents and provide insights into how the AV could have minimized the probability of incurring an accident.
  arXiv  Detail & Related papers  (2021-03-12T17:00:23Z)
• Can Autonomous Vehicles Identify, Recover From, and Adapt to Distribution Shifts? [104.04999499189402]
  Out-of-training-distribution (OOD) scenarios are a common challenge of learning agents at deployment. We propose an uncertainty-aware planning method, called emphrobust imitative planning (RIP) Our method can detect and recover from some distribution shifts, reducing the overconfident and catastrophic extrapolations in OOD scenes. We introduce an autonomous car novel-scene benchmark, textttCARNOVEL, to evaluate the robustness of driving agents to a suite of tasks with distribution shifts.
  arXiv  Detail & Related papers  (2020-06-26T11:07:32Z)
• Scalable Autonomous Vehicle Safety Validation through Dynamic Programming and Scene Decomposition [37.61747231296097]
  We present a new safety validation approach that attempts to estimate the distribution over failures of an autonomous policy using approximate dynamic programming. In both experiments, we observed an increase in the number of failures discovered compared to baseline approaches.
  arXiv  Detail & Related papers  (2020-04-14T21:03:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.