Falsification of Learning-Based Controllers through Multi-Fidelity Bayesian Optimization

• URL: http://arxiv.org/abs/2212.14118v4
• Date: Fri, 28 Apr 2023 20:11:07 GMT
• Title: Falsification of Learning-Based Controllers through Multi-Fidelity Bayesian Optimization
• Authors: Zahra Shahrooei, Mykel J. Kochenderfer, Ali Baheri
• Abstract summary: We propose a multi-fidelity falsification framework using Bayesian optimization. This method allows us to automatically switch between inexpensive, inaccurate information from a low-fidelity simulator and expensive, accurate information from a high-fidelity simulator.
• Score: 34.71695000650056
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Simulation-based falsification is a practical testing method to increase confidence that the system will meet safety requirements. Because full-fidelity simulations can be computationally demanding, we investigate the use of simulators with different levels of fidelity. As a first step, we express the overall safety specification in terms of environmental parameters and structure this safety specification as an optimization problem. We propose a multi-fidelity falsification framework using Bayesian optimization, which is able to determine at which level of fidelity we should conduct a safety evaluation in addition to finding possible instances from the environment that cause the system to fail. This method allows us to automatically switch between inexpensive, inaccurate information from a low-fidelity simulator and expensive, accurate information from a high-fidelity simulator in a cost-effective way. Our experiments on various environments in simulation demonstrate that multi-fidelity Bayesian optimization has falsification performance comparable to single-fidelity Bayesian optimization but with much lower cost.

Related papers

• Optimizing Falsification for Learning-Based Control Systems: A Multi-Fidelity Bayesian Approach [40.58350379106314]
  falsification problem involves the identification of counterexamples that violate system safety requirements. We propose a multi-fidelity Bayesian optimization falsification framework that harnesses simulators with varying levels of accuracy.
  arXiv  Detail & Related papers  (2024-09-12T14:51:03Z)
• Towards Safe Multi-Task Bayesian Optimization [1.3654846342364308]
  Reduced physical models of the system can be incorporated into the optimization process, accelerating it. These models are able to offer an approximation of the actual system, and evaluating them is significantly cheaper. Safety is a crucial criterion for online optimization methods such as Bayesian optimization.
  arXiv  Detail & Related papers  (2023-12-12T13:59:26Z)
• SAVME: Efficient Safety Validation for Autonomous Systems Using Meta-Learning [36.896695278624776]
  We propose a Bayesian approach that integrates meta-learning strategies with a multi-armed bandit framework. We showcase our methodology using a cutting-edge 3D driving simulator, incorporating 16 fidelity settings for an autonomous vehicle stack. Our approach achieves a significant speedup, up to 18 times faster compared to traditional methods.
  arXiv  Detail & Related papers  (2023-09-21T20:41:47Z)
• Near-optimal Policy Identification in Active Reinforcement Learning [84.27592560211909]
  AE-LSVI is a novel variant of the kernelized least-squares value RL (LSVI) algorithm that combines optimism with pessimism for active exploration. We show that AE-LSVI outperforms other algorithms in a variety of environments when robustness to the initial state is required.
  arXiv  Detail & Related papers  (2022-12-19T14:46:57Z)
• Falsification of Cyber-Physical Systems using Bayesian Optimization [4.164131508933521]
  Cyber-physical systems (CPSs) are often complex and safety-critical, making it both challenging and crucial to ensure that the system's specifications are met. Simulation-based falsification is a practical testing technique for increasing confidence in a CPS's correctness, as it only requires that the system be simulated. This study investigates a sample-efficient approach that learns a surrogate model to capture the relationship between input signal parameterization and specification evaluation.
  arXiv  Detail & Related papers  (2022-09-14T15:52:19Z)
• Log Barriers for Safe Black-box Optimization with Application to Safe Reinforcement Learning [72.97229770329214]
  We introduce a general approach for seeking high dimensional non-linear optimization problems in which maintaining safety during learning is crucial. Our approach called LBSGD is based on applying a logarithmic barrier approximation with a carefully chosen step size. We demonstrate the effectiveness of our approach on minimizing violation in policy tasks in safe reinforcement learning.
  arXiv  Detail & Related papers  (2022-07-21T11:14:47Z)
• Learning Robust Output Control Barrier Functions from Safe Expert Demonstrations [50.37808220291108]
  This paper addresses learning safe output feedback control laws from partial observations of expert demonstrations. We first propose robust output control barrier functions (ROCBFs) as a means to guarantee safety. We then formulate an optimization problem to learn ROCBFs from expert demonstrations that exhibit safe system behavior.
  arXiv  Detail & Related papers  (2021-11-18T23:21:00Z)
• Finding Failures in High-Fidelity Simulation using Adaptive Stress Testing and the Backward Algorithm [35.076062292062325]
  Adaptive stress testing (AST) is a method that uses reinforcement learning to find the most likely failure of a system. AST with a deep reinforcement learning solver has been shown to be effective in finding failures across a range of different systems. To improve efficiency, we present a method that first finds failures in a low-fidelity simulator. It then uses the backward algorithm, which trains a deep neural network policy using a single expert demonstration, to adapt the low-fidelity failures to high-fidelity.
  arXiv  Detail & Related papers  (2021-07-27T16:54:04Z)
• AutoSimulate: (Quickly) Learning Synthetic Data Generation [70.82315853981838]
  We propose an efficient alternative for optimal synthetic data generation based on a novel differentiable approximation of the objective. We demonstrate that the proposed method finds the optimal data distribution faster (up to $50times$), with significantly reduced training data generation (up to $30times$) and better accuracy ($+8.7%$) on real-world test datasets than previous methods.
  arXiv  Detail & Related papers  (2020-08-16T11:36:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.