Conformal Prediction Intervals for Markov Decision Process Trajectories

• URL: http://arxiv.org/abs/2206.04860v1
• Date: Fri, 10 Jun 2022 03:43:53 GMT
• Title: Conformal Prediction Intervals for Markov Decision Process Trajectories
• Authors: Thomas G. Dietterich, Jesse Hostetler
• Abstract summary: This paper provides conformal prediction intervals over the future behavior of an autonomous system executing a fixed control policy on a Markov Decision Process (MDP) The method is illustrated on MDPs for invasive species management and StarCraft2 battles.
• Score: 10.68332392039368
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Before delegating a task to an autonomous system, a human operator may want a guarantee about the behavior of the system. This paper extends previous work on conformal prediction for functional data and conformalized quantile regression to provide conformal prediction intervals over the future behavior of an autonomous system executing a fixed control policy on a Markov Decision Process (MDP). The prediction intervals are constructed by applying conformal corrections to prediction intervals computed by quantile regression. The resulting intervals guarantee that with probability $1-\delta$ the observed trajectory will lie inside the prediction interval, where the probability is computed with respect to the starting state distribution and the stochasticity of the MDP. The method is illustrated on MDPs for invasive species management and StarCraft2 battles.

Related papers

• SMURF-THP: Score Matching-based UnceRtainty quantiFication for Transformer Hawkes Process [76.98721879039559]
  We propose SMURF-THP, a score-based method for learning Transformer Hawkes process and quantifying prediction uncertainty. Specifically, SMURF-THP learns the score function of events' arrival time based on a score-matching objective. We conduct extensive experiments in both event type prediction and uncertainty quantification of arrival time.
  arXiv  Detail & Related papers  (2023-10-25T03:33:45Z)
• Confidence Calibration for Systems with Cascaded Predictive Modules [9.393699753285997]
  We present novel solutions based on conformal prediction to provide prediction intervals calibrated for a predictive system consisting of cascaded modules. Our key idea is to leverage module-level validation data to characterize the system-level error distribution without direct access to end-to-end validation data. In comparison to prediction intervals calibrated for individual modules, our solutions generate improved intervals with more accurate performance guarantees for system predictions.
  arXiv  Detail & Related papers  (2023-09-21T22:12:24Z)
• Model Predictive Control with Gaussian-Process-Supported Dynamical Constraints for Autonomous Vehicles [82.65261980827594]
  We propose a model predictive control approach for autonomous vehicles that exploits learned Gaussian processes for predicting human driving behavior. A multi-mode predictive control approach considers the possible intentions of the human drivers.
  arXiv  Detail & Related papers  (2023-03-08T17:14:57Z)
• Will My Robot Achieve My Goals? Predicting the Probability that an MDP Policy Reaches a User-Specified Behavior Target [56.99669411766284]
  As an autonomous system performs a task, it should maintain a calibrated estimate of the probability that it will achieve the user's goal. This paper considers settings where the user's goal is specified as a target interval for a real-valued performance summary. We compute the probability estimates by inverting conformal prediction.
  arXiv  Detail & Related papers  (2022-11-29T18:41:20Z)
• Conformal Quantitative Predictive Monitoring of STL Requirements for Stochastic Processes [4.279881803310469]
  We introduce textitquantitative predictive monitoring (QPM), the first PM method to support processes and rich specifications given in Signal Temporal Logic (STL) QPM provides a quantitative measure of satisfaction by predicting the quantitative (aka robust) STL semantics of $phi$. We show how our monitors can be combined in a compositional manner to handle composite formulas.
  arXiv  Detail & Related papers  (2022-11-04T11:08:29Z)
• A general framework for multi-step ahead adaptive conformal heteroscedastic time series forecasting [0.0]
  This paper introduces a novel model-agnostic algorithm called adaptive ensemble batch multi-input multi-output conformalized quantile regression (AEnbMIMOCQR) It enables forecasters to generate multi-step ahead prediction intervals for a fixed pre-specified miscoverage rate in a distribution-free manner. Our method is grounded on conformal prediction principles, however, it does not require data splitting and provides close to exact coverage even when the data is not exchangeable.
  arXiv  Detail & Related papers  (2022-07-28T16:40:26Z)
• Robust and Adaptive Temporal-Difference Learning Using An Ensemble of Gaussian Processes [70.80716221080118]
  The paper takes a generative perspective on policy evaluation via temporal-difference (TD) learning. The OS-GPTD approach is developed to estimate the value function for a given policy by observing a sequence of state-reward pairs. To alleviate the limited expressiveness associated with a single fixed kernel, a weighted ensemble (E) of GP priors is employed to yield an alternative scheme.
  arXiv  Detail & Related papers  (2021-12-01T23:15:09Z)
• CovarianceNet: Conditional Generative Model for Correct Covariance Prediction in Human Motion Prediction [71.31516599226606]
  We present a new method to correctly predict the uncertainty associated with the predicted distribution of future trajectories. Our approach, CovariaceNet, is based on a Conditional Generative Model with Gaussian latent variables.
  arXiv  Detail & Related papers  (2021-09-07T09:38:24Z)
• Neural Predictive Monitoring under Partial Observability [4.1316328854247155]
  We present a learning-based method for predictive monitoring (PM) that produces accurate and reliable reachability predictions despite partial observability (PO) Our method results in highly accurate reachability predictions and error detection, as well as tight prediction regions with guaranteed coverage.
  arXiv  Detail & Related papers  (2021-08-16T15:08:20Z)
• Prediction Intervals: Split Normal Mixture from Quality-Driven Deep Ensembles [4.521131595149397]
  We present a method for generating prediction intervals along with point estimates from an ensemble of neural networks. We propose a multi-objective loss function fusing quality measures related to prediction intervals and point estimates, and a penalty function, which enforces semantic integrity of the results. Our results show that both our quality-driven loss function and our aggregation method contribute to well-calibrated prediction intervals and point estimates.
  arXiv  Detail & Related papers  (2020-07-19T13:46:34Z)
• Batch Stationary Distribution Estimation [98.18201132095066]
  We consider the problem of approximating the stationary distribution of an ergodic Markov chain given a set of sampled transitions. We propose a consistent estimator that is based on recovering a correction ratio function over the given data.
  arXiv  Detail & Related papers  (2020-03-02T09:10:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.