BehAVExplor: Behavior Diversity Guided Testing for Autonomous Driving Systems

• URL: http://arxiv.org/abs/2307.07493v1
• Date: Fri, 14 Jul 2023 17:24:39 GMT
• Title: BehAVExplor: Behavior Diversity Guided Testing for Autonomous Driving Systems
• Authors: Mingfei Cheng, Yuan Zhou, Xiaofei Xie
• Abstract summary: Testing autonomous driving systems (ADSs) is a critical task for ensuring their reliability and safety. Existing methods mainly focus on searching for safety violations while the diversity of the generated test cases is ignored. We present a novel behavior-guided fuzzing technique (BehAVExplor) to explore the different behaviors of the ego vehicle and detect diverse violations.
• Score: 27.223488110349567
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Testing Autonomous Driving Systems (ADSs) is a critical task for ensuring the reliability and safety of autonomous vehicles. Existing methods mainly focus on searching for safety violations while the diversity of the generated test cases is ignored, which may generate many redundant test cases and failures. Such redundant failures can reduce testing performance and increase failure analysis costs. In this paper, we present a novel behavior-guided fuzzing technique (BehAVExplor) to explore the different behaviors of the ego vehicle (i.e., the vehicle controlled by the ADS under test) and detect diverse violations. Specifically, we design an efficient unsupervised model, called BehaviorMiner, to characterize the behavior of the ego vehicle. BehaviorMiner extracts the temporal features from the given scenarios and performs a clustering-based abstraction to group behaviors with similar features into abstract states. A new test case will be added to the seed corpus if it triggers new behaviors (e.g., cover new abstract states). Due to the potential conflict between the behavior diversity and the general violation feedback, we further propose an energy mechanism to guide the seed selection and the mutation. The energy of a seed quantifies how good it is. We evaluated BehAVExplor on Apollo, an industrial-level ADS, and LGSVL simulation environment. Empirical evaluation results show that BehAVExplor can effectively find more diverse violations than the state-of-the-art.

Related papers

• Indiscriminate Disruption of Conditional Inference on Multivariate Gaussians [60.22542847840578]
  Despite advances in adversarial machine learning, inference for Gaussian models in the presence of an adversary is notably understudied. We consider a self-interested attacker who wishes to disrupt a decisionmaker's conditional inference and subsequent actions by corrupting a set of evidentiary variables. To avoid detection, the attacker also desires the attack to appear plausible wherein plausibility is determined by the density of the corrupted evidence.
  arXiv  Detail & Related papers  (2024-11-21T17:46:55Z)
• Rigorous Simulation-based Testing for Autonomous Driving Systems -- Targeting the Achilles' Heel of Four Open Autopilots [6.229766691427486]
  We propose a rigorous test method based on breaking down scenarios into simple ones. We generate test cases for critical configurations that place the vehicle under test in critical situations. Test cases reveal major defects in Apollo, Autoware, and the Carla and LGSVL autopilots.
  arXiv  Detail & Related papers  (2024-05-27T08:06:21Z)
• PAFOT: A Position-Based Approach for Finding Optimal Tests of Autonomous Vehicles [4.243926243206826]
  This paper proposes PAFOT, a position-based approach testing framework. PAFOT generates adversarial driving scenarios to expose safety violations of Automated Driving Systems. Experiments show PAFOT can effectively generate safety-critical scenarios to crash ADSs and is able to find collisions in a short simulation time.
  arXiv  Detail & Related papers  (2024-05-06T10:04:40Z)
• Interactive Autonomous Navigation with Internal State Inference and Interactivity Estimation [58.21683603243387]
  We propose three auxiliary tasks with relational-temporal reasoning and integrate them into the standard Deep Learning framework. These auxiliary tasks provide additional supervision signals to infer the behavior patterns other interactive agents. Our approach achieves robust and state-of-the-art performance in terms of standard evaluation metrics.
  arXiv  Detail & Related papers  (2023-11-27T18:57:42Z)
• DARTH: Holistic Test-time Adaptation for Multiple Object Tracking [87.72019733473562]
  Multiple object tracking (MOT) is a fundamental component of perception systems for autonomous driving. Despite the urge of safety in driving systems, no solution to the MOT adaptation problem to domain shift in test-time conditions has ever been proposed. We introduce DARTH, a holistic test-time adaptation framework for MOT.
  arXiv  Detail & Related papers  (2023-10-03T10:10:42Z)
• ReMAV: Reward Modeling of Autonomous Vehicles for Finding Likely Failure Events [1.84926694477846]
  We propose a black-box testing framework that uses offline trajectories first to analyze the existing behavior of autonomous vehicles. Our experiment shows an increase in 35, 23, 48, and 50% in the occurrences of vehicle collision, road object collision, pedestrian collision, and offroad steering events.
  arXiv  Detail & Related papers  (2023-08-28T13:09:00Z)
• Boundary State Generation for Testing and Improvement of Autonomous Driving Systems [8.670873561640903]
  We present GENBO, a novel test generator for autonomous driving systems (ADSs) testing. We use such boundary conditions to augment the initial training dataset and retrain the DNN model under test. Our evaluation results show that the retrained model has, on average, up to 3x higher success rate on a separate set of evaluation tracks with respect to the original DNN model.
  arXiv  Detail & Related papers  (2023-07-20T05:07:51Z)
• Driving Anomaly Detection Using Conditional Generative Adversarial Network [26.45460503638333]
  This study proposes an unsupervised method to quantify driving anomalies using a conditional generative adversarial network (GAN) The approach predicts upcoming driving scenarios by conditioning the models on the previously observed signals. The results are validated with perceptual evaluations, where annotators are asked to assess the risk and familiarity of the videos detected with high anomaly scores.
  arXiv  Detail & Related papers  (2022-03-15T22:10:01Z)
• Contingencies from Observations: Tractable Contingency Planning with Learned Behavior Models [82.34305824719101]
  Humans have a remarkable ability to make decisions by accurately reasoning about future events. We develop a general-purpose contingency planner that is learned end-to-end using high-dimensional scene observations. We show how this model can tractably learn contingencies from behavioral observations.
  arXiv  Detail & Related papers  (2021-04-21T14:30:20Z)
• Detecting Rewards Deterioration in Episodic Reinforcement Learning [63.49923393311052]
  In many RL applications, once training ends, it is vital to detect any deterioration in the agent performance as soon as possible. We consider an episodic framework, where the rewards within each episode are not independent, nor identically-distributed, nor Markov. We define the mean-shift in a way corresponding to deterioration of a temporal signal (such as the rewards), and derive a test for this problem with optimal statistical power.
  arXiv  Detail & Related papers  (2020-10-22T12:45:55Z)
• Can Autonomous Vehicles Identify, Recover From, and Adapt to Distribution Shifts? [104.04999499189402]
  Out-of-training-distribution (OOD) scenarios are a common challenge of learning agents at deployment. We propose an uncertainty-aware planning method, called emphrobust imitative planning (RIP) Our method can detect and recover from some distribution shifts, reducing the overconfident and catastrophic extrapolations in OOD scenes. We introduce an autonomous car novel-scene benchmark, textttCARNOVEL, to evaluate the robustness of driving agents to a suite of tasks with distribution shifts.
  arXiv  Detail & Related papers  (2020-06-26T11:07:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.