Parameter-Free and Group Conditional Online Conformal Prediction

• URL: http://arxiv.org/abs/2606.00419v2
• Date: Tue, 02 Jun 2026 15:50:21 GMT
• Title: Parameter-Free and Group Conditional Online Conformal Prediction
• Abstract summary: Uncertainty quantification is critical for the deployment of machine learning predictors in real-world scenarios.<n>Online conformal prediction (OCP) methods address this issue at the expense of either (i) group-wise error control or (ii) learning-rate independent implementation.<n>We propose a parameter-free algorithm for group-conditional OCP and demonstrate that it achieves the best group-conditional coverage guarantees.
• Score: 15.407762755896677
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Uncertainty quantification (UQ) is critical for the deployment of machine learning predictors in real-world scenarios where the data distribution may shift over time (i.e., data may not be exchangeable). Online conformal prediction (OCP) methods address this issue at the expense of either (i) group-wise error control or (ii) learning-rate independent implementation. Group-conditional coverage is essential for fairness across different collections of data points and for providing finer UQ guarantees. Parameter-free optimization is crucial for robustness to adversarial and unknown data shifts. We propose a parameter-free algorithm for group-conditional OCP and demonstrate that it achieves the best group-conditional coverage guarantees. We evaluate our algorithm on synthetic and real-world data, demonstrating that our method not only improves the reliability of existing parameter-free OCP methods but also provides prediction intervals that are comparable in size to well-tuned group-conditional approaches. By unifying group-conditional coverage with parameter-free online algorithms, our work lays a foundation for fair and robust uncertainty quantification in shifting environments.

Related papers

• Conditional Predictive Inference for General Structured Data with Group Symmetries [0.43328790010087576]
  C-SymmPI is a framework that achieves near-conditional coverage under general data structures with group symmetries.<n>Inspired by relaxed multi-accuracy, our approach reformulates conditional coverage as miscoverage error over a user-specified function class.<n>For computational efficiency, we develop two variants: a projection-based algorithm for high-dimensional observations, and a sampling-based algorithm for large or infinite groups.
  arXiv  Detail & Related papers  (2026-05-18T06:41:20Z)
• Fair Conformal Classification via Learning Representation-Based Groups [18.926790668827937]
  This paper introduces a fair conformal inference framework for classification tasks.<n>By balancing effectiveness and efficiency in producing compact, informative prediction sets, our approach paves a practical pathway toward trustworthy machine learning.
  arXiv  Detail & Related papers  (2026-05-12T14:37:17Z)
• Conformal Prediction Assessment: A Framework for Conditional Coverage Evaluation and Selection [8.778189690708578]
  Conformal Prediction Assessment (CPA) is a framework that reframes the evaluation of conditional coverage as a supervised learning task.<n> Conditional Validity Index (CVI) decomposes reliability into safety (undercoverage risk) and efficiency (overcoverage cost)
  arXiv  Detail & Related papers  (2026-03-28T08:30:17Z)
• COIN: Uncertainty-Guarding Selective Question Answering for Foundation Models with Provable Risk Guarantees [51.5976496056012]
  COIN is an uncertainty-guarding selection framework that calibrates statistically valid thresholds to filter a single generated answer per question.<n>COIN estimates the empirical error rate on a calibration set and applies confidence interval methods to establish a high-probability upper bound on the true error rate.<n>We demonstrate COIN's robustness in risk control, strong test-time power in retaining admissible answers, and predictive efficiency under limited calibration data.
  arXiv  Detail & Related papers  (2025-06-25T07:04:49Z)
• Rectifying Conformity Scores for Better Conditional Coverage [75.73184036344908]
  We present a new method for generating confidence sets within the split conformal prediction framework.<n>Our method performs a trainable transformation of any given conformity score to improve conditional coverage while ensuring exact marginal coverage.
  arXiv  Detail & Related papers  (2025-02-22T19:54:14Z)
• Noise-Adaptive Conformal Classification with Marginal Coverage [53.74125453366155]
  We introduce an adaptive conformal inference method capable of efficiently handling deviations from exchangeability caused by random label noise.<n>We validate our method through extensive numerical experiments demonstrating its effectiveness on synthetic and real data sets.
  arXiv  Detail & Related papers  (2025-01-29T23:55:23Z)
• Adaptive Conformal Inference by Betting [51.272991377903274]
  We consider the problem of adaptive conformal inference without any assumptions about the data generating process.<n>Existing approaches for adaptive conformal inference are based on optimizing the pinball loss using variants of online gradient descent.<n>We propose a different approach for adaptive conformal inference that leverages parameter-free online convex optimization techniques.
  arXiv  Detail & Related papers  (2024-12-26T18:42:08Z)
• Adjusting Regression Models for Conditional Uncertainty Calibration [46.69079637538012]
  We propose a novel algorithm to train a regression function to improve the conditional coverage after applying the split conformal prediction procedure. We establish an upper bound for the miscoverage gap between the conditional coverage and the nominal coverage rate and propose an end-to-end algorithm to control this upper bound.
  arXiv  Detail & Related papers  (2024-09-26T01:55:45Z)
• Probabilistic Conformal Prediction with Approximate Conditional Validity [81.30551968980143]
  We develop a new method for generating prediction sets that combines the flexibility of conformal methods with an estimate of the conditional distribution. Our method consistently outperforms existing approaches in terms of conditional coverage.
  arXiv  Detail & Related papers  (2024-07-01T20:44:48Z)
• Stratified Prediction-Powered Inference for Hybrid Language Model Evaluation [62.2436697657307]
  Prediction-powered inference (PPI) is a method that improves statistical estimates based on limited human-labeled data.<n>We propose a method called Stratified Prediction-Powered Inference (StratPPI)<n>We show that the basic PPI estimates can be considerably improved by employing simple data stratification strategies.
  arXiv  Detail & Related papers  (2024-06-06T17:37:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.