TARGET: Automated Scenario Generation from Traffic Rules for Testing Autonomous Vehicles

• URL: http://arxiv.org/abs/2305.06018v2
• Date: Sun, 8 Oct 2023 09:08:46 GMT
• Title: TARGET: Automated Scenario Generation from Traffic Rules for Testing Autonomous Vehicles
• Authors: Yao Deng, Jiaohong Yao, Zhi Tu, Xi Zheng, Mengshi Zhang, Tianyi Zhang
• Abstract summary: TARGET is an end-to-end framework designed for the automatic generation of test scenarios grounded in traffic rules. We leverage a large language model to automatically extract knowledge from traffic rules and convert the traffic rule descriptions to DSL representations. TARGET synthesizes executable test scenario scripts to render the testing scenarios in a simulator.
• Score: 8.508687759145841
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Ensuring the safety and robustness of autonomous driving systems (ADSs) is imperative. One of the crucial methods towards this assurance is the meticulous construction and execution of test scenarios, a task often regarded as tedious and laborious. In response to this challenge, this paper introduces TARGET, an end-to-end framework designed for the automatic generation of test scenarios grounded in established traffic rules. Specifically, we design a domain-specific language (DSL) with concise and expressive syntax for scenario descriptions. To handle the natural language complexity and ambiguity in traffic rule descriptions, we leverage a large language model to automatically extract knowledge from traffic rules and convert the traffic rule descriptions to DSL representations. Based on these representations, TARGET synthesizes executable test scenario scripts to render the testing scenarios in a simulator. Comprehensive evaluations of the framework were conducted on four distinct ADSs, yielding a total of 217 test scenarios spread across eight diverse maps. These scenarios identify approximately 700 rule violations, collisions, and other significant issues, including navigation failures. Moreover, for each detected anomaly, TARGET provides detailed scenario recordings and log reports, significantly easing the process of troubleshooting and root cause analysis. Two of these causes have been confirmed by the ADS developers; one is corroborated by an existing bug report from the ADS, and the other one is attributed to the limited functionality of the ADS.

Related papers

• From Accidents to Insights: Leveraging Multimodal Data for Scenario-Driven ADS Testing [3.984220091774453]
  This paper introduces TRACE, a scenario-based ADS Test case Generation framework for Critical Scenarios. By leveraging multimodal data to extract challenging scenarios from real-world car crash reports, TRACE constructs numerous critical test cases with less data. User feedback reveals that TRACE demonstrates superior scenario reconstruction accuracy, with 77.5% of the scenarios being rated as'mostly or 'totally' consistent.
  arXiv  Detail & Related papers  (2025-02-04T05:21:29Z)
• Black-Box Adversarial Attack on Vision Language Models for Autonomous Driving [65.61999354218628]
  We take the first step toward designing black-box adversarial attacks specifically targeting vision-language models (VLMs) in autonomous driving systems. We propose Cascading Adversarial Disruption (CAD), which targets low-level reasoning breakdown by generating and injecting semantics. We present Risky Scene Induction, which addresses dynamic adaptation by leveraging a surrogate VLM to understand and construct high-level risky scenarios.
  arXiv  Detail & Related papers  (2025-01-23T11:10:02Z)
• LMM-enhanced Safety-Critical Scenario Generation for Autonomous Driving System Testing From Non-Accident Traffic Videos [22.638869562921133]
  It is paramount to generate a diverse range of safety-critical test scenarios for autonomous driving systems. Some accident-free real-world scenarios can not only lead to misbehaviors in ADSs but also be leveraged for the generation of ADS violations. It is of significant importance to discover safety violations of ADSs from routine traffic scenarios.
  arXiv  Detail & Related papers  (2024-06-16T09:05:56Z)
• REDriver: Runtime Enforcement for Autonomous Vehicles [6.97499033700151]
  We propose REDriver, a general and modular approach to runtime enforcement of autonomous driving systems. ReDriver monitors the planned trajectory of the ADS based on a quantitative semantics of STL. It uses a gradient-driven algorithm to repair the trajectory when a violation of the specification is likely.
  arXiv  Detail & Related papers  (2024-01-04T13:08:38Z)
• DARTH: Holistic Test-time Adaptation for Multiple Object Tracking [87.72019733473562]
  Multiple object tracking (MOT) is a fundamental component of perception systems for autonomous driving. Despite the urge of safety in driving systems, no solution to the MOT adaptation problem to domain shift in test-time conditions has ever been proposed. We introduce DARTH, a holistic test-time adaptation framework for MOT.
  arXiv  Detail & Related papers  (2023-10-03T10:10:42Z)
• Clustering-based Criticality Analysis for Testing of Automated Driving Systems [0.18416014644193066]
  This paper focuses on the the goal to reduce the scenario set by clustering concrete scenarios from a single logical scenario. By employing clustering techniques, redundant and uninteresting scenarios can be identified and eliminated, resulting in a representative scenario set.
  arXiv  Detail & Related papers  (2023-06-22T08:36:20Z)
• DeepAccident: A Motion and Accident Prediction Benchmark for V2X Autonomous Driving [76.29141888408265]
  We propose a large-scale dataset containing diverse accident scenarios that frequently occur in real-world driving. The proposed DeepAccident dataset includes 57K annotated frames and 285K annotated samples, approximately 7 times more than the large-scale nuScenes dataset.
  arXiv  Detail & Related papers  (2023-04-03T17:37:00Z)
• ADC: Adversarial attacks against object Detection that evade Context consistency checks [55.8459119462263]
  We show that even context consistency checks can be brittle to properly crafted adversarial examples. We propose an adaptive framework to generate examples that subvert such defenses. Our results suggest that how to robustly model context and check its consistency, is still an open problem.
  arXiv  Detail & Related papers  (2021-10-24T00:25:09Z)
• Generating and Characterizing Scenarios for Safety Testing of Autonomous Vehicles [86.9067793493874]
  We propose efficient mechanisms to characterize and generate testing scenarios using a state-of-the-art driving simulator. We use our method to characterize real driving data from the Next Generation Simulation (NGSIM) project. We rank the scenarios by defining metrics based on the complexity of avoiding accidents and provide insights into how the AV could have minimized the probability of incurring an accident.
  arXiv  Detail & Related papers  (2021-03-12T17:00:23Z)
• Detecting 32 Pedestrian Attributes for Autonomous Vehicles [103.87351701138554]
  In this paper, we address the problem of jointly detecting pedestrians and recognizing 32 pedestrian attributes. We introduce a Multi-Task Learning (MTL) model relying on a composite field framework, which achieves both goals in an efficient way. We show competitive detection and attribute recognition results, as well as a more stable MTL training.
  arXiv  Detail & Related papers  (2020-12-04T15:10:12Z)
• Pass-Fail Criteria for Scenario-Based Testing of Automated Driving Systems [0.0]
  This paper sets out a framework for assessing an automated driving system's behavioural safety in normal operation. Risk-based rules cannot give a pass/fail decision from a single test case. This considers statistical performance across many individual tests.
  arXiv  Detail & Related papers  (2020-05-19T13:13:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.