Tree-Based Scenario Classification: A Formal Framework for Coverage Analysis on Test Drives of Autonomous Vehicles

• URL: http://arxiv.org/abs/2307.05106v1
• Date: Tue, 11 Jul 2023 08:30:57 GMT
• Title: Tree-Based Scenario Classification: A Formal Framework for Coverage Analysis on Test Drives of Autonomous Vehicles
• Authors: Till Schallau, Stefan Naujokat, Fiona Kullmann, Falk Howar
• Abstract summary: In scenario-based testing, relevant (driving) scenarios are the basis of tests. We address the open challenges of classifying sets of scenarios and measuring coverage of theses scenarios in recorded test drives.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Scenario-based testing is envisioned as a key approach for the safety assurance of autonomous vehicles. In scenario-based testing, relevant (driving) scenarios are the basis of tests. Many recent works focus on specification, variation, generation and execution of individual scenarios. In this work, we address the open challenges of classifying sets of scenarios and measuring coverage of theses scenarios in recorded test drives. Technically, we define logic-based classifiers that compute features of scenarios on complex data streams and combine these classifiers into feature trees that describe sets of scenarios. We demonstrate the expressiveness and effectiveness of our approach by defining a scenario classifier for urban driving and evaluating it on data recorded from simulations.

Related papers

• Generating Out-Of-Distribution Scenarios Using Language Models [58.47597351184034]
  Large Language Models (LLMs) have shown promise in autonomous driving. This paper introduces a framework for generating diverse Out-Of-Distribution (OOD) driving scenarios. We evaluate our framework through extensive simulations and introduce a new "OOD-ness" metric.
  arXiv  Detail & Related papers  (2024-11-25T16:38:17Z)
• Towards a Completeness Argumentation for Scenario Concepts [0.2184775414778289]
  This paper argues a sufficient completeness of a scenario concept using a goal structured notation. Methods are applied to a scenario concept and the inD dataset to prove the usability.
  arXiv  Detail & Related papers  (2024-04-02T13:29:38Z)
• Generative Judge for Evaluating Alignment [84.09815387884753]
  We propose a generative judge with 13B parameters, Auto-J, designed to address these challenges. Our model is trained on user queries and LLM-generated responses under massive real-world scenarios. Experimentally, Auto-J outperforms a series of strong competitors, including both open-source and closed-source models.
  arXiv  Detail & Related papers  (2023-10-09T07:27:15Z)
• Clustering-based Criticality Analysis for Testing of Automated Driving Systems [0.18416014644193066]
  This paper focuses on the the goal to reduce the scenario set by clustering concrete scenarios from a single logical scenario. By employing clustering techniques, redundant and uninteresting scenarios can be identified and eliminated, resulting in a representative scenario set.
  arXiv  Detail & Related papers  (2023-06-22T08:36:20Z)
• UMSE: Unified Multi-scenario Summarization Evaluation [52.60867881867428]
  Summarization quality evaluation is a non-trivial task in text summarization. We propose Unified Multi-scenario Summarization Evaluation Model (UMSE) Our UMSE is the first unified summarization evaluation framework engaged with the ability to be used in three evaluation scenarios.
  arXiv  Detail & Related papers  (2023-05-26T12:54:44Z)
• Generating Useful Accident-Prone Driving Scenarios via a Learned Traffic Prior [135.78858513845233]
  STRIVE is a method to automatically generate challenging scenarios that cause a given planner to produce undesirable behavior, like collisions. To maintain scenario plausibility, the key idea is to leverage a learned model of traffic motion in the form of a graph-based conditional VAE. A subsequent optimization is used to find a "solution" to the scenario, ensuring it is useful to improve the given planner.
  arXiv  Detail & Related papers  (2021-12-09T18:03:27Z)
• Realistic simulation of users for IT systems in cyber ranges [63.20765930558542]
  We instrument each machine by means of an external agent to generate user activity. This agent combines both deterministic and deep learning based methods to adapt to different environment. We also propose conditional text generation models to facilitate the creation of conversations and documents.
  arXiv  Detail & Related papers  (2021-11-23T10:53:29Z)
• Addressing the IEEE AV Test Challenge with Scenic and VerifAI [10.221093591444731]
  This paper summarizes our formal approach to testing autonomous vehicles (AVs) in simulation for the IEEE AV Test Challenge. We demonstrate a systematic testing framework leveraging our previous work on formally-driven simulation for intelligent cyber-physical systems.
  arXiv  Detail & Related papers  (2021-08-20T04:51:27Z)
• Generating and Characterizing Scenarios for Safety Testing of Autonomous Vehicles [86.9067793493874]
  We propose efficient mechanisms to characterize and generate testing scenarios using a state-of-the-art driving simulator. We use our method to characterize real driving data from the Next Generation Simulation (NGSIM) project. We rank the scenarios by defining metrics based on the complexity of avoiding accidents and provide insights into how the AV could have minimized the probability of incurring an accident.
  arXiv  Detail & Related papers  (2021-03-12T17:00:23Z)
• A Scenario-Based Development Framework for Autonomous Driving [0.0]
  This article summarizes the research progress of scenario-based testing and development technology for autonomous vehicles. We propose the definition of scenario, the elements of the scenario, the data source of the scenario, the processing method of the scenario data, and scenario-based V-Model.
  arXiv  Detail & Related papers  (2020-11-03T03:06:48Z)
• Real-World Scenario Mining for the Assessment of Automated Vehicles [12.962830182937035]
  We propose a new method to capture scenarios from real-world data using a two-step approach. The method is not specific for one type of scenario and, therefore, it can be applied to a large variety of scenarios.
  arXiv  Detail & Related papers  (2020-05-31T10:10:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.