Failure Probability Estimation for Black-Box Autonomous Systems using State-Dependent Importance Sampling Proposals

• URL: http://arxiv.org/abs/2412.02154v1
• Date: Tue, 03 Dec 2024 04:28:58 GMT
• Title: Failure Probability Estimation for Black-Box Autonomous Systems using State-Dependent Importance Sampling Proposals
• Authors: Harrison Delecki, Sydney M. Katz, Mykel J. Kochenderfer,
• Abstract summary: Estimating the probability of failure is a critical step in developing safety-critical autonomous systems. Direct estimation methods such as Monte Carlo sampling are often impractical due to the rarity of failures in these systems. We propose an adaptive importance sampling algorithm to address these limitations.
• Score: 37.579437595742995
• License:
• Abstract: Estimating the probability of failure is a critical step in developing safety-critical autonomous systems. Direct estimation methods such as Monte Carlo sampling are often impractical due to the rarity of failures in these systems. Existing importance sampling approaches do not scale to sequential decision-making systems with large state spaces and long horizons. We propose an adaptive importance sampling algorithm to address these limitations. Our method minimizes the forward Kullback-Leibler divergence between a state-dependent proposal distribution and a relaxed form of the optimal importance sampling distribution. Our method uses Markov score ascent methods to estimate this objective. We evaluate our approach on four sequential systems and show that it provides more accurate failure probability estimates than baseline Monte Carlo and importance sampling techniques. This work is open sourced.

Related papers

• Enhanced Importance Sampling through Latent Space Exploration in Normalizing Flows [69.8873421870522]
  importance sampling is a rare event simulation technique used in Monte Carlo simulations. We propose a method for more efficient sampling by updating the proposal distribution in the latent space of a normalizing flow.
  arXiv  Detail & Related papers  (2025-01-06T21:18:02Z)
• Probabilistic Parameter Estimators and Calibration Metrics for Pose Estimation from Image Features [30.85393323542915]
  This paper addresses the challenge of probabilistic parameter estimation given measurement uncertainty in real-time. We present three probabilistic parameter estimators: a least-squares sampling approach, a linear approximation method, and a probabilistic programming estimator. We demonstrate that the linear approximation estimator can produce sharp and well-calibrated pose predictions significantly faster than the other methods but may yield overconfident predictions in certain scenarios.
  arXiv  Detail & Related papers  (2024-07-23T07:02:01Z)
• Model-based Validation as Probabilistic Inference [37.61747231296097]
  Estimating the distribution over failures is a key step in validating autonomous systems. We frame estimating the distribution over failure trajectories for sequential systems as Bayesian inference. Our approach is demonstrated in an inverted pendulum control system, an autonomous vehicle driving scenario, and a partially observable lunar lander.
  arXiv  Detail & Related papers  (2023-05-17T03:27:36Z)
• A Tale of Sampling and Estimation in Discounted Reinforcement Learning [50.43256303670011]
  We present a minimax lower bound on the discounted mean estimation problem. We show that estimating the mean by directly sampling from the discounted kernel of the Markov process brings compelling statistical properties.
  arXiv  Detail & Related papers  (2023-04-11T09:13:17Z)
• Uncertainty Estimation based on Geometric Separation [13.588210692213568]
  In machine learning, accurately predicting the probability that a specific input is correct is crucial for risk management. We put forward a novel geometric-based approach for improving uncertainty estimations in machine learning models.
  arXiv  Detail & Related papers  (2023-01-11T13:19:24Z)
• A Deep Reinforcement Learning Approach to Rare Event Estimation [30.670114229970526]
  An important step in the design of autonomous systems is to evaluate the probability that a failure will occur. In safety-critical domains, the failure probability is extremely small so that the evaluation of a policy through Monte Carlo sampling is inefficient. We develop two adaptive importance sampling algorithms that can efficiently estimate the probability of rare events for sequential decision making systems.
  arXiv  Detail & Related papers  (2022-11-22T18:29:14Z)
• Case Studies for Computing Density of Reachable States for Safe Autonomous Motion Planning [8.220217498103313]
  Density of the reachable states can help understand the risk of safety-critical systems. Recent work provides a data-driven approach to compute the density distribution of autonomous systems' forward reachable states online. In this paper, we study the use of such approach in combination with model predictive control for verifiable safe path planning under uncertainties.
  arXiv  Detail & Related papers  (2022-09-16T17:38:24Z)
• Sampling-Based Robust Control of Autonomous Systems with Non-Gaussian Noise [59.47042225257565]
  We present a novel planning method that does not rely on any explicit representation of the noise distributions. First, we abstract the continuous system into a discrete-state model that captures noise by probabilistic transitions between states. We capture these bounds in the transition probability intervals of a so-called interval Markov decision process (iMDP)
  arXiv  Detail & Related papers  (2021-10-25T06:18:55Z)
• Stein Variational Model Predictive Control [130.60527864489168]
  Decision making under uncertainty is critical to real-world, autonomous systems. Model Predictive Control (MPC) methods have demonstrated favorable performance in practice, but remain limited when dealing with complex distributions. We show that this framework leads to successful planning in challenging, non optimal control problems.
  arXiv  Detail & Related papers  (2020-11-15T22:36:59Z)
• Batch Stationary Distribution Estimation [98.18201132095066]
  We consider the problem of approximating the stationary distribution of an ergodic Markov chain given a set of sampled transitions. We propose a consistent estimator that is based on recovering a correction ratio function over the given data.
  arXiv  Detail & Related papers  (2020-03-02T09:10:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.