Evolving Testing Scenario Generation Method and Intelligence Evaluation Framework for Automated Vehicles

• URL: http://arxiv.org/abs/2306.07142v1
• Date: Mon, 12 Jun 2023 14:26:12 GMT
• Title: Evolving Testing Scenario Generation Method and Intelligence Evaluation Framework for Automated Vehicles
• Authors: Yining Ma, Wei Jiang, Lingtong Zhang, Junyi Chen, Hong Wang, Chen Lv, Xuesong Wang, Lu Xiong
• Abstract summary: This paper proposes an evolving scenario generation method that utilizes deep reinforcement learning (DRL) to create human-like BVs for testing and intelligence evaluation of automated vehicles (AVs) The results demonstrate that the proposed evolving scenario exhibits the highest level of complexity compared to other baseline scenarios and has more than 85% similarity to naturalistic driving data.
• Score: 12.670180834651912
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Interaction between the background vehicles (BVs) and automated vehicles (AVs) in scenario-based testing plays a critical role in evaluating the intelligence of the AVs. Current testing scenarios typically employ predefined or scripted BVs, which inadequately reflect the complexity of human-like social behaviors in real-world driving scenarios, and also lack a systematic metric for evaluating the comprehensive intelligence of AVs. Therefore, this paper proposes an evolving scenario generation method that utilizes deep reinforcement learning (DRL) to create human-like BVs for testing and intelligence evaluation of AVs. Firstly, a class of driver models with human-like competitive, cooperative, and mutual driving motivations is designed. Then, utilizing an improved "level-k" training procedure, the three distinct driver models acquire game-based interactive driving policies. And these models are assigned to BVs for generating evolving scenarios in which all BVs can interact continuously and evolve diverse contents. Next, a framework including safety, driving efficiency, and interaction utility are presented to evaluate and quantify the intelligence performance of 3 systems under test (SUTs), indicating the effectiveness of the evolving scenario for intelligence testing. Finally, the complexity and fidelity of the proposed evolving testing scenario are validated. The results demonstrate that the proposed evolving scenario exhibits the highest level of complexity compared to other baseline scenarios and has more than 85% similarity to naturalistic driving data. This highlights the potential of the proposed method to facilitate the development and evaluation of high-level AVs in a realistic and challenging environment.

Related papers

• Generative Diffusion-based Contract Design for Efficient AI Twins Migration in Vehicular Embodied AI Networks [55.15079732226397]
  Embodied AI is a rapidly advancing field that bridges the gap between cyberspace and physical space. In VEANET, embodied AI twins act as in-vehicle AI assistants to perform diverse tasks supporting autonomous driving.
  arXiv  Detail & Related papers  (2024-10-02T02:20:42Z)
• Work-in-Progress: Crash Course: Can (Under Attack) Autonomous Driving Beat Human Drivers? [60.51287814584477]
  This paper evaluates the inherent risks in autonomous driving by examining the current landscape of AVs. We develop specific claims highlighting the delicate balance between the advantages of AVs and potential security challenges in real-world scenarios.
  arXiv  Detail & Related papers  (2024-05-14T09:42:21Z)
• A Cognitive-Based Trajectory Prediction Approach for Autonomous Driving [21.130543517747995]
  This paper introduces the Human-Like Trajectory Prediction (H) model, which adopts a teacher-student knowledge distillation framework. The "teacher" model mimics the visual processing of the human brain, particularly the functions of the occipital and temporal lobes. The "student" model focuses on real-time interaction and decision-making, capturing essential perceptual cues for accurate prediction.
  arXiv  Detail & Related papers  (2024-02-29T15:22:26Z)
• Interactive Autonomous Navigation with Internal State Inference and Interactivity Estimation [58.21683603243387]
  We propose three auxiliary tasks with relational-temporal reasoning and integrate them into the standard Deep Learning framework. These auxiliary tasks provide additional supervision signals to infer the behavior patterns other interactive agents. Our approach achieves robust and state-of-the-art performance in terms of standard evaluation metrics.
  arXiv  Detail & Related papers  (2023-11-27T18:57:42Z)
• Automatic Generation of Scenarios for System-level Simulation-based Verification of Autonomous Driving Systems [0.0]
  This paper describes the framework for system-level simulation-based V&V of autonomous systems using AI components. The framework is based on a simulation model of the system, an abstract model that describes symbolically the system behavior. Various coverage criteria can be defined to guide the automated generation of the scenarios.
  arXiv  Detail & Related papers  (2023-11-16T11:03:13Z)
• Stackelberg Driver Model for Continual Policy Improvement in Scenario-Based Closed-Loop Autonomous Driving [5.765939495779461]
  adversarial generation methods have emerged as a class of efficient approaches to synthesize safety-critical scenarios. We tailor the Stackelberg Driver Model (SDM) to accurately characterize the hierarchical nature of vehicle interaction dynamics. Our algorithm exhibits superior performance compared to several baselines especially in higher dimensional scenarios.
  arXiv  Detail & Related papers  (2023-09-25T15:47:07Z)
• Learning Driver Models for Automated Vehicles via Knowledge Sharing and Personalization [2.07180164747172]
  This paper describes a framework for learning Automated Vehicles (AVs) driver models via knowledge sharing between vehicles and personalization. It finds several applications across transportation engineering including intelligent transportation systems, traffic management, and vehicle-to-vehicle communication.
  arXiv  Detail & Related papers  (2023-08-31T17:18:15Z)
• A Study of Situational Reasoning for Traffic Understanding [63.45021731775964]
  We devise three novel text-based tasks for situational reasoning in the traffic domain. We adopt four knowledge-enhanced methods that have shown generalization capability across language reasoning tasks in prior work. We provide in-depth analyses of model performance on data partitions and examine model predictions categorically.
  arXiv  Detail & Related papers  (2023-06-05T01:01:12Z)
• Generating and Characterizing Scenarios for Safety Testing of Autonomous Vehicles [86.9067793493874]
  We propose efficient mechanisms to characterize and generate testing scenarios using a state-of-the-art driving simulator. We use our method to characterize real driving data from the Next Generation Simulation (NGSIM) project. We rank the scenarios by defining metrics based on the complexity of avoiding accidents and provide insights into how the AV could have minimized the probability of incurring an accident.
  arXiv  Detail & Related papers  (2021-03-12T17:00:23Z)
• Fine-Grained Vehicle Perception via 3D Part-Guided Visual Data Augmentation [77.60050239225086]
  We propose an effective training data generation process by fitting a 3D car model with dynamic parts to vehicles in real images. Our approach is fully automatic without any human interaction. We present a multi-task network for VUS parsing and a multi-stream network for VHI parsing.
  arXiv  Detail & Related papers  (2020-12-15T03:03:38Z)
• Building Trust in Autonomous Vehicles: Role of Virtual Reality Driving Simulators in HMI Design [8.39368916644651]
  We propose a methodology to validate the user experience in AVs based on continuous, objective information gathered from physiological signals. We applied this methodology to the design of a head-up display interface delivering visual cues about the vehicle's sensory and planning systems.
  arXiv  Detail & Related papers  (2020-07-27T08:42:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.