A Generalizable Physics-informed Learning Framework for Risk Probability
Estimation
- URL: http://arxiv.org/abs/2305.06432v2
- Date: Thu, 4 Jan 2024 17:19:39 GMT
- Title: A Generalizable Physics-informed Learning Framework for Risk Probability
Estimation
- Authors: Zhuoyuan Wang, Yorie Nakahira
- Abstract summary: We develop an efficient method to evaluate the probabilities of long-term risk and their gradients.
The proposed method exploits the fact that long-term risk probability satisfies certain partial differential equations.
Numerical results show the proposed method has better sample efficiency, generalizes well to unseen regions, and can adapt to systems with changing parameters.
- Score: 1.8855270809505869
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Accurate estimates of long-term risk probabilities and their gradients are
critical for many stochastic safe control methods. However, computing such risk
probabilities in real-time and in unseen or changing environments is
challenging. Monte Carlo (MC) methods cannot accurately evaluate the
probabilities and their gradients as an infinitesimal devisor can amplify the
sampling noise. In this paper, we develop an efficient method to evaluate the
probabilities of long-term risk and their gradients. The proposed method
exploits the fact that long-term risk probability satisfies certain partial
differential equations (PDEs), which characterize the neighboring relations
between the probabilities, to integrate MC methods and physics-informed neural
networks. We provide theoretical guarantees of the estimation error given
certain choices of training configurations. Numerical results show the proposed
method has better sample efficiency, generalizes well to unseen regions, and
can adapt to systems with changing parameters. The proposed method can also
accurately estimate the gradients of risk probabilities, which enables first-
and second-order techniques on risk probabilities to be used for learning and
control.
Related papers
- Generalizable Physics-Informed Learning for Stochastic Safety-Critical Systems [8.277567852741244]
We propose an efficient method to evaluate long-term risk probabilities and their gradients using short-term samples without sufficient risk events.
We show in simulation that the proposed technique has improved sample efficiency, generalizes well to unseen regions, and adapts to changing system parameters.
arXiv Detail & Related papers (2024-07-11T21:10:03Z) - Myopically Verifiable Probabilistic Certificates for Safe Control and Learning [7.6918726072590555]
In environments, set invariance-based methods that restrict the probability of risk events in infinitesimal time intervals may exhibit significant long-term risks.
On the other hand, reachability-based approaches that account for the long-term future may require prohibitive in real-time decision making.
arXiv Detail & Related papers (2024-04-23T20:29:01Z) - Data-Adaptive Tradeoffs among Multiple Risks in Distribution-Free Prediction [55.77015419028725]
We develop methods that permit valid control of risk when threshold and tradeoff parameters are chosen adaptively.
Our methodology supports monotone and nearly-monotone risks, but otherwise makes no distributional assumptions.
arXiv Detail & Related papers (2024-03-28T17:28:06Z) - Physics-informed RL for Maximal Safety Probability Estimation [0.8287206589886881]
We study how to estimate the long-term safety probability of maximally safe actions without sufficient coverage of samples from risky states and long-term trajectories.
The proposed method can also estimate long-term risk using short-term samples and deduce the risk of unsampled states.
arXiv Detail & Related papers (2024-03-25T03:13:56Z) - Model-Based Epistemic Variance of Values for Risk-Aware Policy
Optimization [63.32053223422317]
We consider the problem of quantifying uncertainty over expected cumulative rewards in model-based reinforcement learning.
In particular, we focus on characterizing the variance over values induced by a distribution over MDPs.
We propose a new uncertainty Bellman equation (UBE) whose solution converges to the true posterior variance over values.
arXiv Detail & Related papers (2023-12-07T15:55:58Z) - Likelihood Ratio Confidence Sets for Sequential Decision Making [51.66638486226482]
We revisit the likelihood-based inference principle and propose to use likelihood ratios to construct valid confidence sequences.
Our method is especially suitable for problems with well-specified likelihoods.
We show how to provably choose the best sequence of estimators and shed light on connections to online convex optimization.
arXiv Detail & Related papers (2023-11-08T00:10:21Z) - Mind the Uncertainty: Risk-Aware and Actively Exploring Model-Based
Reinforcement Learning [26.497229327357935]
We introduce a simple but effective method for managing risk in model-based reinforcement learning with trajectory sampling.
Experiments indicate that the separation of uncertainties is essential to performing well with data-driven approaches in uncertain and safety-critical control environments.
arXiv Detail & Related papers (2023-09-11T16:10:58Z) - Risk Consistent Multi-Class Learning from Label Proportions [64.0125322353281]
This study addresses a multiclass learning from label proportions (MCLLP) setting in which training instances are provided in bags.
Most existing MCLLP methods impose bag-wise constraints on the prediction of instances or assign them pseudo-labels.
A risk-consistent method is proposed for instance classification using the empirical risk minimization framework.
arXiv Detail & Related papers (2022-03-24T03:49:04Z) - Quantifying Uncertainty in Deep Spatiotemporal Forecasting [67.77102283276409]
We describe two types of forecasting problems: regular grid-based and graph-based.
We analyze UQ methods from both the Bayesian and the frequentist point view, casting in a unified framework via statistical decision theory.
Through extensive experiments on real-world road network traffic, epidemics, and air quality forecasting tasks, we reveal the statistical computational trade-offs for different UQ methods.
arXiv Detail & Related papers (2021-05-25T14:35:46Z) - Probabilistic robust linear quadratic regulators with Gaussian processes [73.0364959221845]
Probabilistic models such as Gaussian processes (GPs) are powerful tools to learn unknown dynamical systems from data for subsequent use in control design.
We present a novel controller synthesis for linearized GP dynamics that yields robust controllers with respect to a probabilistic stability margin.
arXiv Detail & Related papers (2021-05-17T08:36:18Z) - Handling Epistemic and Aleatory Uncertainties in Probabilistic Circuits [18.740781076082044]
We propose an approach to overcome the independence assumption behind most of the approaches dealing with a large class of probabilistic reasoning.
We provide an algorithm for Bayesian learning from sparse, albeit complete, observations.
Each leaf of such circuits is labelled with a beta-distributed random variable that provides us with an elegant framework for representing uncertain probabilities.
arXiv Detail & Related papers (2021-02-22T10:03:15Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.