Adaptive Coverage Policies in Conformal Prediction
- URL: http://arxiv.org/abs/2510.04318v1
- Date: Sun, 05 Oct 2025 18:37:22 GMT
- Title: Adaptive Coverage Policies in Conformal Prediction
- Authors: Etienne Gauthier, Francis Bach, Michael I. Jordan,
- Abstract summary: We leverage advances in e-values and post-hoc conformal inference to improve conformal prediction methods.<n>We propose to optimize an adaptive coverage policy by training a neural network using a leave-one-out procedure on a calibration set.
- Score: 46.298122008420414
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Traditional conformal prediction methods construct prediction sets such that the true label falls within the set with a user-specified coverage level. However, poorly chosen coverage levels can result in uninformative predictions, either producing overly conservative sets when the coverage level is too high, or empty sets when it is too low. Moreover, the fixed coverage level cannot adapt to the specific characteristics of each individual example, limiting the flexibility and efficiency of these methods. In this work, we leverage recent advances in e-values and post-hoc conformal inference, which allow the use of data-dependent coverage levels while maintaining valid statistical guarantees. We propose to optimize an adaptive coverage policy by training a neural network using a leave-one-out procedure on the calibration set, allowing the coverage level and the resulting prediction set size to vary with the difficulty of each individual example. We support our approach with theoretical coverage guarantees and demonstrate its practical benefits through a series of experiments.
Related papers
- Conditional Coverage Diagnostics for Conformal Prediction [47.93989136542648]
We show that conditional coverage estimation can be a classification problem.<n>We call the resulting family of metrics excess risk of the target coverage (ERT)<n>We release an open-source package for ERT as well as previous conditional coverage metrics.
arXiv Detail & Related papers (2025-12-12T18:47:39Z) - Distribution-informed Online Conformal Prediction [53.674678995825666]
We propose Conformal Optimistic Prediction (COP), an online conformal prediction algorithm incorporating underlying data pattern into the update rule.<n>COP produces tighter prediction sets when predictable pattern exists, while retaining valid coverage guarantees even when estimates are inaccurate.<n>We prove that COP can achieve valid coverage and construct shorter prediction intervals than other baselines.
arXiv Detail & Related papers (2025-12-08T17:51:49Z) - Aggregating Conformal Prediction Sets via α-Allocation [22.63151097928744]
This work introduces a principled aggregation strategy that optimally allocates confidence levels across multiple conformal prediction sets.<n>Experiments on synthetic and real-world datasets demonstrate that COLA achieves considerably smaller prediction sets than state-of-the-art baselines.
arXiv Detail & Related papers (2025-11-15T07:11:21Z) - Probabilistic Conformal Coverage Guarantees in Small-Data Settings [0.02648566468224904]
Conformal prediction provides distribution-free prediction sets with guaranteed marginal coverage.<n>In split conformal prediction this guarantee is training-conditional only in expectation.<n>This variance undermines effective risk control in practical applications.
arXiv Detail & Related papers (2025-09-18T18:41:50Z) - COIN: Uncertainty-Guarding Selective Question Answering for Foundation Models with Provable Risk Guarantees [51.5976496056012]
COIN is an uncertainty-guarding selection framework that calibrates statistically valid thresholds to filter a single generated answer per question.<n>COIN estimates the empirical error rate on a calibration set and applies confidence interval methods to establish a high-probability upper bound on the true error rate.<n>We demonstrate COIN's robustness in risk control, strong test-time power in retaining admissible answers, and predictive efficiency under limited calibration data.
arXiv Detail & Related papers (2025-06-25T07:04:49Z) - Semi-Supervised Conformal Prediction With Unlabeled Nonconformity Score [19.15617038007535]
Conformal prediction (CP) is a powerful framework for uncertainty quantification.<n>In real-world applications where labeled data is often limited, standard CP can lead to coverage deviation and output overly large prediction sets.<n>We propose SemiCP, leveraging both labeled data and unlabeled data for calibration.
arXiv Detail & Related papers (2025-05-27T12:57:44Z) - Adaptive Temperature Scaling with Conformal Prediction [47.51764759462074]
We propose to the best of our knowledge the first method for assigning calibrated probabilities to elements of a conformal prediction set.<n>Our approach frames this as an adaptive calibration problem, selecting an input-specific temperature parameter to match the desired coverage level.
arXiv Detail & Related papers (2025-05-21T12:18:15Z) - Rectifying Conformity Scores for Better Conditional Coverage [75.73184036344908]
We present a new method for generating confidence sets within the split conformal prediction framework.<n>Our method performs a trainable transformation of any given conformity score to improve conditional coverage while ensuring exact marginal coverage.
arXiv Detail & Related papers (2025-02-22T19:54:14Z) - Conformal Prediction Sets with Improved Conditional Coverage using Trust Scores [52.92618442300405]
It is impossible to achieve exact, distribution-free conditional coverage in finite samples.<n>We propose an alternative conformal prediction algorithm that targets coverage where it matters most.
arXiv Detail & Related papers (2025-01-17T12:01:56Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.