Finding Failures in High-Fidelity Simulation using Adaptive Stress Testing and the Backward Algorithm

• URL: http://arxiv.org/abs/2107.12940v1
• Date: Tue, 27 Jul 2021 16:54:04 GMT
• Title: Finding Failures in High-Fidelity Simulation using Adaptive Stress Testing and the Backward Algorithm
• Authors: Mark Koren and Ahmed Nassar and Mykel J. Kochenderfer
• Abstract summary: Adaptive stress testing (AST) is a method that uses reinforcement learning to find the most likely failure of a system. AST with a deep reinforcement learning solver has been shown to be effective in finding failures across a range of different systems. To improve efficiency, we present a method that first finds failures in a low-fidelity simulator. It then uses the backward algorithm, which trains a deep neural network policy using a single expert demonstration, to adapt the low-fidelity failures to high-fidelity.
• Score: 35.076062292062325
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Validating the safety of autonomous systems generally requires the use of high-fidelity simulators that adequately capture the variability of real-world scenarios. However, it is generally not feasible to exhaustively search the space of simulation scenarios for failures. Adaptive stress testing (AST) is a method that uses reinforcement learning to find the most likely failure of a system. AST with a deep reinforcement learning solver has been shown to be effective in finding failures across a range of different systems. This approach generally involves running many simulations, which can be very expensive when using a high-fidelity simulator. To improve efficiency, we present a method that first finds failures in a low-fidelity simulator. It then uses the backward algorithm, which trains a deep neural network policy using a single expert demonstration, to adapt the low-fidelity failures to high-fidelity. We have created a series of autonomous vehicle validation case studies that represent some of the ways low-fidelity and high-fidelity simulators can differ, such as time discretization. We demonstrate in a variety of case studies that this new AST approach is able to find failures with significantly fewer high-fidelity simulation steps than are needed when just running AST directly in high-fidelity. As a proof of concept, we also demonstrate AST on NVIDIA's DriveSim simulator, an industry state-of-the-art high-fidelity simulator for finding failures in autonomous vehicles.

Related papers

• Autonomous Vehicle Controllers From End-to-End Differentiable Simulation [60.05963742334746]
  We propose a differentiable simulator and design an analytic policy gradients (APG) approach to training AV controllers. Our proposed framework brings the differentiable simulator into an end-to-end training loop, where gradients of environment dynamics serve as a useful prior to help the agent learn a more grounded policy. We find significant improvements in performance and robustness to noise in the dynamics, as well as overall more intuitive human-like handling.
  arXiv  Detail & Related papers  (2024-09-12T11:50:06Z)
• The Power of Resets in Online Reinforcement Learning [73.64852266145387]
  We explore the power of simulators through online reinforcement learning with local simulator access (or, local planning) We show that MDPs with low coverability can be learned in a sample-efficient fashion with only $Qstar$-realizability. We show that the notorious Exogenous Block MDP problem is tractable under local simulator access.
  arXiv  Detail & Related papers  (2024-04-23T18:09:53Z)
• Bridging the Sim-to-Real Gap with Bayesian Inference [53.61496586090384]
  We present SIM-FSVGD for learning robot dynamics from data. We use low-fidelity physical priors to regularize the training of neural network models. We demonstrate the effectiveness of SIM-FSVGD in bridging the sim-to-real gap on a high-performance RC racecar system.
  arXiv  Detail & Related papers  (2024-03-25T11:29:32Z)
• Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Research [76.93956925360638]
  Waymax is a new data-driven simulator for autonomous driving in multi-agent scenes. It runs entirely on hardware accelerators such as TPUs/GPUs and supports in-graph simulation for training. We benchmark a suite of popular imitation and reinforcement learning algorithms with ablation studies on different design decisions.
  arXiv  Detail & Related papers  (2023-10-12T20:49:15Z)
• SAVME: Efficient Safety Validation for Autonomous Systems Using Meta-Learning [36.896695278624776]
  We propose a Bayesian approach that integrates meta-learning strategies with a multi-armed bandit framework. We showcase our methodology using a cutting-edge 3D driving simulator, incorporating 16 fidelity settings for an autonomous vehicle stack. Our approach achieves a significant speedup, up to 18 times faster compared to traditional methods.
  arXiv  Detail & Related papers  (2023-09-21T20:41:47Z)
• Adaptive Failure Search Using Critical States from Domain Experts [9.93890332477992]
  Failure search may be done through logging substantial vehicle miles in either simulation or real world testing. AST is one such method that poses the problem of failure search as a Markov decision process. We show that the incorporation of critical states into the AST framework generates failure scenarios with increased safety violations.
  arXiv  Detail & Related papers  (2023-04-01T18:14:41Z)
• Simulation-to-reality UAV Fault Diagnosis with Deep Learning [20.182411473467656]
  We propose a deep learning model that addresses the simulation-to-reality gap in fault diagnosis of quadrotors. Our proposed approach achieves an accuracy of 96% in detecting propeller faults. This is the first reliable and efficient method for simulation-to-reality fault diagnosis of quadrotor propellers.
  arXiv  Detail & Related papers  (2023-02-09T02:37:48Z)
• Accelerated Policy Learning with Parallel Differentiable Simulation [59.665651562534755]
  We present a differentiable simulator and a new policy learning algorithm (SHAC) Our algorithm alleviates problems with local minima through a smooth critic function. We show substantial improvements in sample efficiency and wall-clock time over state-of-the-art RL and differentiable simulation-based algorithms.
  arXiv  Detail & Related papers  (2022-04-14T17:46:26Z)
• Efficient and Effective Generation of Test Cases for Pedestrian Detection -- Search-based Software Testing of Baidu Apollo in SVL [14.482670650074885]
  This paper presents a study on testing pedestrian detection and emergency braking system of the Baidu Apollo autonomous driving platform within the SVL simulator. We propose an evolutionary automated test generation technique that generates failure-revealing scenarios for Apollo in the SVL environment. In order to demonstrate the efficiency and effectiveness of our approach, we also report the results from a baseline random generation technique.
  arXiv  Detail & Related papers  (2021-09-16T13:11:53Z)
• Towards Automated Safety Coverage and Testing for Autonomous Vehicles with Reinforcement Learning [0.3683202928838613]
  Validation puts the autonomous vehicle system to the test in scenarios or situations that the system would likely encounter in everyday driving. We propose using reinforcement learning (RL) to generate failure examples and unexpected traffic situations for the AV software implementation.
  arXiv  Detail & Related papers  (2020-05-22T19:00:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.