Online conformal prediction with decaying step sizes

• URL: http://arxiv.org/abs/2402.01139v2
• Date: Tue, 28 May 2024 14:17:22 GMT
• Title: Online conformal prediction with decaying step sizes
• Authors: Anastasios N. Angelopoulos, Rina Foygel Barber, Stephen Bates,
• Abstract summary: We introduce a method for online conformal prediction with decaying step sizes. Unlike previous methods, we can simultaneously estimate a population quantile when it exists.
• Score: 15.884682750072399
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce a method for online conformal prediction with decaying step sizes. Like previous methods, ours possesses a retrospective guarantee of coverage for arbitrary sequences. However, unlike previous methods, we can simultaneously estimate a population quantile when it exists. Our theory and experiments indicate substantially improved practical properties: in particular, when the distribution is stable, the coverage is close to the desired level for every time point, not just on average over the observed sequence.

Related papers

• Rectifying Conformity Scores for Better Conditional Coverage [75.73184036344908]
  We present a new method for generating confidence sets within the split conformal prediction framework. Our method performs a trainable transformation of any given conformity score to improve conditional coverage while ensuring exact marginal coverage.
  arXiv  Detail & Related papers  (2025-02-22T19:54:14Z)
• Conformal Generative Modeling with Improved Sample Efficiency through Sequential Greedy Filtering [55.15192437680943]
  Generative models lack rigorous statistical guarantees for their outputs. We propose a sequential conformal prediction method producing prediction sets that satisfy a rigorous statistical guarantee. This guarantee states that with high probability, the prediction sets contain at least one admissible (or valid) example.
  arXiv  Detail & Related papers  (2024-10-02T15:26:52Z)
• Conformalized Interval Arithmetic with Symmetric Calibration [9.559062601251464]
  We develop conformal prediction intervals for single target to the prediction interval for sum of multiple targets. We show that our method outperforms existing conformalized approaches as well as non-conformal approaches.
  arXiv  Detail & Related papers  (2024-08-20T15:27:18Z)
• Probabilistic Conformal Prediction with Approximate Conditional Validity [81.30551968980143]
  We develop a new method for generating prediction sets that combines the flexibility of conformal methods with an estimate of the conditional distribution. Our method consistently outperforms existing approaches in terms of conditional coverage.
  arXiv  Detail & Related papers  (2024-07-01T20:44:48Z)
• Conditional validity of heteroskedastic conformal regression [12.905195278168506]
  Conformal prediction and split conformal prediction offer a distribution-free approach to estimating prediction intervals with statistical guarantees. Recent work has shown that split conformal prediction can produce state-of-the-art prediction intervals when focusing on marginal coverage. This paper tries to shed new light on how prediction intervals can be constructed, using methods such as normalized and Mondrian conformal prediction.
  arXiv  Detail & Related papers  (2023-09-15T11:10:46Z)
• Improved Online Conformal Prediction via Strongly Adaptive Online Learning [86.4346936885507]
  We develop new online conformal prediction methods that minimize the strongly adaptive regret. We prove that our methods achieve near-optimal strongly adaptive regret for all interval lengths simultaneously. Experiments show that our methods consistently obtain better coverage and smaller prediction sets than existing methods on real-world tasks.
  arXiv  Detail & Related papers  (2023-02-15T18:59:30Z)
• Sequential Predictive Conformal Inference for Time Series [16.38369532102931]
  We present a new distribution-free conformal prediction algorithm for sequential data (e.g., time series) We specifically account for the nature that time series data are non-exchangeable, and thus many existing conformal prediction algorithms are not applicable.
  arXiv  Detail & Related papers  (2022-12-07T05:07:27Z)
• Predictive Inference with Feature Conformal Prediction [80.77443423828315]
  We propose feature conformal prediction, which extends the scope of conformal prediction to semantic feature spaces. From a theoretical perspective, we demonstrate that feature conformal prediction provably outperforms regular conformal prediction under mild assumptions. Our approach could be combined with not only vanilla conformal prediction, but also other adaptive conformal prediction methods.
  arXiv  Detail & Related papers  (2022-10-01T02:57:37Z)
• Near-optimal inference in adaptive linear regression [60.08422051718195]
  Even simple methods like least squares can exhibit non-normal behavior when data is collected in an adaptive manner. We propose a family of online debiasing estimators to correct these distributional anomalies in at least squares estimation. We demonstrate the usefulness of our theory via applications to multi-armed bandit, autoregressive time series estimation, and active learning with exploration.
  arXiv  Detail & Related papers  (2021-07-05T21:05:11Z)
• Valid prediction intervals for regression problems [12.905195278168506]
  We review the above four classes of methods from a conceptual and experimental point of view. Results on benchmark data sets from various domains highlight large fluctuations in performance from one data set to another. We illustrate how conformal prediction can be used as a general calibration procedure for methods that deliver poor results without a calibration step.
  arXiv  Detail & Related papers  (2021-07-01T10:59:36Z)
• Batch Stationary Distribution Estimation [98.18201132095066]
  We consider the problem of approximating the stationary distribution of an ergodic Markov chain given a set of sampled transitions. We propose a consistent estimator that is based on recovering a correction ratio function over the given data.
  arXiv  Detail & Related papers  (2020-03-02T09:10:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.