Foundation Models for Rapid Autonomy Validation

• URL: http://arxiv.org/abs/2411.03328v1
• Date: Tue, 22 Oct 2024 15:32:43 GMT
• Title: Foundation Models for Rapid Autonomy Validation
• Authors: Alec Farid, Peter Schleede, Aaron Huang, Christoffer Heckman,
• Abstract summary: A key challenge is that an autonomous vehicle requires testing in every kind of driving scenario it could encounter. We propose the use of a behavior foundation model, specifically a masked autoencoder (MAE), trained to reconstruct driving scenarios.
• Score: 4.417336418010182
• License:
• Abstract: We are motivated by the problem of autonomous vehicle performance validation. A key challenge is that an autonomous vehicle requires testing in every kind of driving scenario it could encounter, including rare events, to provide a strong case for safety and show there is no edge-case pathological behavior. Autonomous vehicle companies rely on potentially millions of miles driven in realistic simulation to expose the driving stack to enough miles to estimate rates and severity of collisions. To address scalability and coverage, we propose the use of a behavior foundation model, specifically a masked autoencoder (MAE), trained to reconstruct driving scenarios. We leverage the foundation model in two complementary ways: we (i) use the learned embedding space to group qualitatively similar scenarios together and (ii) fine-tune the model to label scenario difficulty based on the likelihood of a collision upon re-simulation. We use the difficulty scoring as importance weighting for the groups of scenarios. The result is an approach which can more rapidly estimate the rates and severity of collisions by prioritizing hard scenarios while ensuring exposure to every kind of driving scenario.

Related papers

• CRASH: Challenging Reinforcement-Learning Based Adversarial Scenarios For Safety Hardening [16.305837225117607]
  This paper introduces CRASH - Challenging Reinforcement-learning based Adversarial scenarios for Safety Hardening. First CRASH can control adversarial Non Player Character (NPC) agents in an AV simulator to automatically induce collisions with the Ego vehicle. We also propose a novel approach, that we term safety hardening, which iteratively refines the motion planner by simulating improvement scenarios against adversarial agents.
  arXiv  Detail & Related papers  (2024-11-26T00:00:27Z)
• ICSFuzz: Collision Detector Bug Discovery in Autonomous Driving Simulators [11.343198884451166]
  This paper aims to systematically discover ignored collision scenarios to improve the reliability of autonomous driving simulators. We present ICSFuzz, a black-box fuzzing approach to discover ignored collision scenarios efficiently. We compare ICSFuzz with DriveFuzz, a state-of-the-art simulation-based ADS testing method, by replacing its oracle with our ignored-collision-aware oracle.
  arXiv  Detail & Related papers  (2024-08-11T04:48:54Z)
• SAFE-SIM: Safety-Critical Closed-Loop Traffic Simulation with Diffusion-Controllable Adversaries [94.84458417662407]
  We introduce SAFE-SIM, a controllable closed-loop safety-critical simulation framework. Our approach yields two distinct advantages: 1) generating realistic long-tail safety-critical scenarios that closely reflect real-world conditions, and 2) providing controllable adversarial behavior for more comprehensive and interactive evaluations. We validate our framework empirically using the nuScenes and nuPlan datasets across multiple planners, demonstrating improvements in both realism and controllability.
  arXiv  Detail & Related papers  (2023-12-31T04:14:43Z)
• ReMAV: Reward Modeling of Autonomous Vehicles for Finding Likely Failure Events [1.84926694477846]
  We propose a black-box testing framework that uses offline trajectories first to analyze the existing behavior of autonomous vehicles. Our experiment shows an increase in 35, 23, 48, and 50% in the occurrences of vehicle collision, road object collision, pedestrian collision, and offroad steering events.
  arXiv  Detail & Related papers  (2023-08-28T13:09:00Z)
• DeepAccident: A Motion and Accident Prediction Benchmark for V2X Autonomous Driving [76.29141888408265]
  We propose a large-scale dataset containing diverse accident scenarios that frequently occur in real-world driving. The proposed DeepAccident dataset includes 57K annotated frames and 285K annotated samples, approximately 7 times more than the large-scale nuScenes dataset.
  arXiv  Detail & Related papers  (2023-04-03T17:37:00Z)
• Generating Useful Accident-Prone Driving Scenarios via a Learned Traffic Prior [135.78858513845233]
  STRIVE is a method to automatically generate challenging scenarios that cause a given planner to produce undesirable behavior, like collisions. To maintain scenario plausibility, the key idea is to leverage a learned model of traffic motion in the form of a graph-based conditional VAE. A subsequent optimization is used to find a "solution" to the scenario, ensuring it is useful to improve the given planner.
  arXiv  Detail & Related papers  (2021-12-09T18:03:27Z)
• Generating and Characterizing Scenarios for Safety Testing of Autonomous Vehicles [86.9067793493874]
  We propose efficient mechanisms to characterize and generate testing scenarios using a state-of-the-art driving simulator. We use our method to characterize real driving data from the Next Generation Simulation (NGSIM) project. We rank the scenarios by defining metrics based on the complexity of avoiding accidents and provide insights into how the AV could have minimized the probability of incurring an accident.
  arXiv  Detail & Related papers  (2021-03-12T17:00:23Z)
• AdvSim: Generating Safety-Critical Scenarios for Self-Driving Vehicles [76.46575807165729]
  We propose AdvSim, an adversarial framework to generate safety-critical scenarios for any LiDAR-based autonomy system. By simulating directly from sensor data, we obtain adversarial scenarios that are safety-critical for the full autonomy stack.
  arXiv  Detail & Related papers  (2021-01-16T23:23:12Z)
• Can Autonomous Vehicles Identify, Recover From, and Adapt to Distribution Shifts? [104.04999499189402]
  Out-of-training-distribution (OOD) scenarios are a common challenge of learning agents at deployment. We propose an uncertainty-aware planning method, called emphrobust imitative planning (RIP) Our method can detect and recover from some distribution shifts, reducing the overconfident and catastrophic extrapolations in OOD scenes. We introduce an autonomous car novel-scene benchmark, textttCARNOVEL, to evaluate the robustness of driving agents to a suite of tasks with distribution shifts.
  arXiv  Detail & Related papers  (2020-06-26T11:07:32Z)
• Towards Automated Safety Coverage and Testing for Autonomous Vehicles with Reinforcement Learning [0.3683202928838613]
  Validation puts the autonomous vehicle system to the test in scenarios or situations that the system would likely encounter in everyday driving. We propose using reinforcement learning (RL) to generate failure examples and unexpected traffic situations for the AV software implementation.
  arXiv  Detail & Related papers  (2020-05-22T19:00:38Z)
• Scalable Autonomous Vehicle Safety Validation through Dynamic Programming and Scene Decomposition [37.61747231296097]
  We present a new safety validation approach that attempts to estimate the distribution over failures of an autonomous policy using approximate dynamic programming. In both experiments, we observed an increase in the number of failures discovered compared to baseline approaches.
  arXiv  Detail & Related papers  (2020-04-14T21:03:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.