Time-uniform conformal and PAC prediction

• URL: http://arxiv.org/abs/2602.06297v1
• Date: Fri, 06 Feb 2026 01:41:10 GMT
• Title: Time-uniform conformal and PAC prediction
• Authors: Kayla E. Scharfstein, Arun Kumar Kuchibhotla,
• Abstract summary: We develop an extension of the conformal prediction and related probably approximately correct (PAC) prediction frameworks to sequential settings.<n>The resulting prediction sets are anytime-valid in that their expected coverage is at the required level at any time chosen by the analyst.<n>We present theoretical guarantees for our proposed methods and demonstrate their validity and utility on simulated and real datasets.
• Score: 0.8021197489470758
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Given that machine learning algorithms are increasingly being deployed to aid in high stakes decision-making, uncertainty quantification methods that wrap around these black box models such as conformal prediction have received much attention in recent years. In sequential settings, where data are observed/generated in a streaming fashion, traditional conformal methods do not provide any guarantee without fixing the sample size. More importantly, traditional conformal methods cannot cope with sequentially updated predictions. As such, we develop an extension of the conformal prediction and related probably approximately correct (PAC) prediction frameworks to sequential settings where the number of data points is not fixed in advance. The resulting prediction sets are anytime-valid in that their expected coverage is at the required level at any time chosen by the analyst even if this choice depends on the data. We present theoretical guarantees for our proposed methods and demonstrate their validity and utility on simulated and real datasets.

Related papers

• Distribution-informed Online Conformal Prediction [53.674678995825666]
  We propose Conformal Optimistic Prediction (COP), an online conformal prediction algorithm incorporating underlying data pattern into the update rule.<n>COP produces tighter prediction sets when predictable pattern exists, while retaining valid coverage guarantees even when estimates are inaccurate.<n>We prove that COP can achieve valid coverage and construct shorter prediction intervals than other baselines.
  arXiv  Detail & Related papers  (2025-12-08T17:51:49Z)
• A Gentle Introduction to Conformal Time Series Forecasting [0.0]
  This review unifies recent advances in conformal forecasting methods specifically designed to address nonexchangeable data.<n>We first present a theoretical foundation, deriving finite-sample guarantees for split-conformal prediction under mild weak-dependence conditions.<n>We then survey and classify state-of-the-art approaches that mitigate serial dependence by reweighting calibration data, dynamically updating residual distributions, or adaptively tuning target coverage levels in real time.
  arXiv  Detail & Related papers  (2025-11-17T17:12:51Z)
• Zono-Conformal Prediction: Zonotope-Based Uncertainty Quantification for Regression and Classification Tasks [41.14877380522394]
  Conformal prediction is used to augment a base predictor with prediction sets with statistically valid coverage guarantees.<n>We introduce zono-conformal prediction, a novel approach inspired by interval predictor models and reachset-conformant identification.<n>We show that zono-conformal predictors are less conservative than interval predictor models and standard conformal prediction methods.
  arXiv  Detail & Related papers  (2025-08-14T19:03:28Z)
• Optimal Conformal Prediction under Epistemic Uncertainty [61.46247583794497]
  Conformal prediction (CP) is a popular framework for representing uncertainty.<n>We introduce Bernoulli prediction sets (BPS) which produce the smallest prediction sets that ensure conditional coverage.<n>When given first-order predictions, BPS reduces to the well-known adaptive prediction sets (APS)
  arXiv  Detail & Related papers  (2025-05-25T08:32:44Z)
• Relational Conformal Prediction for Correlated Time Series [56.59852921638328]
  We address the problem of uncertainty quantification in time series by exploiting correlated sequences.<n>We propose a novel distribution-free approach based on conformal prediction framework and quantile regression.<n>Our approach provides accurate coverage and achieves state-of-the-art uncertainty quantification in relevant benchmarks.
  arXiv  Detail & Related papers  (2025-02-13T16:12:17Z)
• Calibrated Probabilistic Forecasts for Arbitrary Sequences [58.54729945445505]
  Real-world data streams can change unpredictably due to distribution shifts, feedback loops and adversarial actors.<n>We present a forecasting framework ensuring valid uncertainty estimates regardless of how data evolves.
  arXiv  Detail & Related papers  (2024-09-27T21:46:42Z)
• Robust Conformal Prediction Using Privileged Information [17.886554223172517]
  We develop a method to generate prediction sets with a guaranteed coverage rate that is robust to corruptions in the training data.<n>Our approach builds on conformal prediction, a powerful framework to construct prediction sets that are valid under the i.i.d assumption.
  arXiv  Detail & Related papers  (2024-06-08T08:56:47Z)
• Sequential Predictive Conformal Inference for Time Series [16.38369532102931]
  We present a new distribution-free conformal prediction algorithm for sequential data (e.g., time series) We specifically account for the nature that time series data are non-exchangeable, and thus many existing conformal prediction algorithms are not applicable.
  arXiv  Detail & Related papers  (2022-12-07T05:07:27Z)
• Private Prediction Sets [72.75711776601973]
  Machine learning systems need reliable uncertainty quantification and protection of individuals' privacy. We present a framework that treats these two desiderata jointly. We evaluate the method on large-scale computer vision datasets.
  arXiv  Detail & Related papers  (2021-02-11T18:59:11Z)
• AutoCP: Automated Pipelines for Accurate Prediction Intervals [84.16181066107984]
  This paper proposes an AutoML framework called Automatic Machine Learning for Conformal Prediction (AutoCP) Unlike the familiar AutoML frameworks that attempt to select the best prediction model, AutoCP constructs prediction intervals that achieve the user-specified target coverage rate. We tested AutoCP on a variety of datasets and found that it significantly outperforms benchmark algorithms.
  arXiv  Detail & Related papers  (2020-06-24T23:13:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.