LAMBDA: Covering the Multimodal Critical Scenarios for Automated Driving Systems by Search Space Quantization

• URL: http://arxiv.org/abs/2412.00517v1
• Date: Sat, 30 Nov 2024 15:57:05 GMT
• Title: LAMBDA: Covering the Multimodal Critical Scenarios for Automated Driving Systems by Search Space Quantization
• Authors: Xinzheng Wu, Junyi Chen, Xingyu Xing, Jian Sun, Ye Tian, Lihao Liu, Yong Shen,
• Abstract summary: Black-Box Optimization (BBO) was introduced to accelerate the scenario-based test of automated driving systems (ADSs) All the subspaces representing danger in the logical scenario space, rather than only the most critical concrete scenario, play a more significant role for the safety evaluation. We propose LAMBDA (Latent-Action Monte-Carlo Beam Search with Density Adaption) to solve BBC problems.
• Score: 33.87626198349963
• License:
• Abstract: Scenario-based virtual testing is one of the most significant methods to test and evaluate the safety of automated driving systems (ADSs). However, it is impractical to enumerate all concrete scenarios in a logical scenario space and test them exhaustively. Recently, Black-Box Optimization (BBO) was introduced to accelerate the scenario-based test of ADSs by utilizing the historical test information to generate new test cases. However, a single optimum found by the BBO algorithm is insufficient for the purpose of a comprehensive safety evaluation of ADSs in a logical scenario. In fact, all the subspaces representing danger in the logical scenario space, rather than only the most critical concrete scenario, play a more significant role for the safety evaluation. Covering as many of the critical concrete scenarios in a logical scenario space through a limited number of tests is defined as the Black-Box Coverage (BBC) problem in this paper. We formalized this problem in a sample-based search paradigm and constructed a coverage criterion with Confusion Matrix Analysis. Furthermore, we propose LAMBDA (Latent-Action Monte-Carlo Beam Search with Density Adaption) to solve BBC problems. LAMBDA can quickly focus on critical subspaces by recursively partitioning the logical scenario space into accepted and rejected parts. Compared with its predecessor LaMCTS, LAMBDA introduces sampling density to overcome the sampling bias from optimization and Beam Search to obtain more parallelizability. Experimental results show that LAMBDA achieves state-of-the-art performance among all baselines and can reach at most 33 and 6000 times faster than Random Search to get 95% coverage of the critical areas in 2- and 5-dimensional synthetic functions, respectively. Experiments also demonstrate that LAMBDA has a promising future in the safety evaluation of ADSs in virtual tests.

Related papers

• Black-Box Adversarial Attack on Vision Language Models for Autonomous Driving [65.61999354218628]
  We take the first step toward designing black-box adversarial attacks specifically targeting vision-language models (VLMs) in autonomous driving systems. We propose Cascading Adversarial Disruption (CAD), which targets low-level reasoning breakdown by generating and injecting semantics. We present Risky Scene Induction, which addresses dynamic adaptation by leveraging a surrogate VLM to understand and construct high-level risky scenarios.
  arXiv  Detail & Related papers  (2025-01-23T11:10:02Z)
• Make Full Use of Testing Information: An Integrated Accelerated Testing and Evaluation Method for Autonomous Driving Systems [6.065650382599096]
  This paper proposes an Integrated accelerated Testing and Evaluation Method (ITEM) for testing and evaluation of autonomous driving systems (ADSs) To make full use of testing information, this paper proposes an Integrated accelerated Testing and Evaluation Method (ITEM) The experimental results show that ITEM could well identify the hazardous domains in both low- and high-dimensional cases, regardless of the shape of the hazardous domains.
  arXiv  Detail & Related papers  (2025-01-21T06:59:25Z)
• UniTTA: Unified Benchmark and Versatile Framework Towards Realistic Test-Time Adaptation [66.05528698010697]
  Test-Time Adaptation aims to adapt pre-trained models to the target domain during testing. Researchers have identified various challenging scenarios and developed diverse methods to address these challenges. We propose a Unified Test-Time Adaptation benchmark, which is comprehensive and widely applicable.
  arXiv  Detail & Related papers  (2024-07-29T15:04:53Z)
• Active Test-Time Adaptation: Theoretical Analyses and An Algorithm [51.84691955495693]
  Test-time adaptation (TTA) addresses distribution shifts for streaming test data in unsupervised settings. We propose the novel problem setting of active test-time adaptation (ATTA) that integrates active learning within the fully TTA setting.
  arXiv  Detail & Related papers  (2024-04-07T22:31:34Z)
• Better Practices for Domain Adaptation [62.70267990659201]
  Domain adaptation (DA) aims to provide frameworks for adapting models to deployment data without using labels. Unclear validation protocol for DA has led to bad practices in the literature. We show challenges across all three branches of domain adaptation methodology.
  arXiv  Detail & Related papers  (2023-09-07T17:44:18Z)
• SOTIF-Compliant Scenario Generation Using Semi-Concrete Scenarios and Parameter Sampling [6.195203785530687]
  SOTIF standard requires scenario-based testing to verify and validate Advanced Driver Assistance Systems and Automated Driving Systems. Existing scenario generation approaches either focus on exploring or exploiting the scenario space. This paper proposes semi-concrete scenarios and parameter sampling to generate SOTIF-compliant test suites.
  arXiv  Detail & Related papers  (2023-08-14T09:25:24Z)
• Realistic Safety-critical Scenarios Search for Autonomous Driving System via Behavior Tree [8.286351881735191]
  We propose the Matrix-Fuzzer, a behavior tree-based testing framework, to automatically generate realistic safety-critical test scenarios. Our approach is able to find the most types of safety-critical scenarios, but only generating around 30% of the total scenarios compared with the baseline algorithm.
  arXiv  Detail & Related papers  (2023-05-11T06:53:03Z)
• LAMBDA: Covering the Solution Set of Black-Box Inequality by Search Space Quantization [1.345821655503426]
  Black-box functions are broadly used to model complex problems that provide no explicit information but the input and output. Covering as much as possible of the solution set through limited evaluations to the black-box objective function is defined as the Black-Box Coverage (BBC) problem.
  arXiv  Detail & Related papers  (2022-03-25T15:24:05Z)
• Projection & Probability-Driven Black-Box Attack [205.9923346080908]
  Existing black-box attacks suffer from the need for excessive queries in the high-dimensional space. We propose Projection & Probability-driven Black-box Attack (PPBA) to tackle this problem. Our method requires at most 24% fewer queries with a higher attack success rate compared with state-of-the-art approaches.
  arXiv  Detail & Related papers  (2020-05-08T03:37:50Z)
• Efficient statistical validation with edge cases to evaluate Highly Automated Vehicles [6.198523595657983]
  The widescale deployment of Autonomous Vehicles seems to be imminent despite many safety challenges that are yet to be resolved. Existing standards focus on deterministic processes where the validation requires only a set of test cases that cover the requirements. This paper presents a new approach to compute the statistical characteristics of a system's behaviour by biasing automatically generated test cases towards the worst case scenarios.
  arXiv  Detail & Related papers  (2020-03-04T04:35:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.