Related papers: Valid Selection among Conformal Sets

Valid Selection among Conformal Sets

URL: http://arxiv.org/abs/2506.20173v1
Date: Wed, 25 Jun 2025 06:59:55 GMT
Title: Valid Selection among Conformal Sets
Authors: Mahmoud Hegazy, Liviu Aolaritei, Michael I. Jordan, Aymeric Dieuleveut,
Abstract summary: Conformal prediction offers a distribution-free framework for constructing prediction sets with coverage guarantees.<n> selecting the most desirable set, such as the smallest, can invalidate the coverage guarantees.<n>We propose a stability-based approach that ensures coverage for the selected prediction set.
Score: 53.016786692105796
License: http://creativecommons.org/licenses/by/4.0/
Abstract: Conformal prediction offers a distribution-free framework for constructing prediction sets with coverage guarantees. In practice, multiple valid conformal prediction sets may be available, arising from different models or methodologies. However, selecting the most desirable set, such as the smallest, can invalidate the coverage guarantees. To address this challenge, we propose a stability-based approach that ensures coverage for the selected prediction set. We extend our results to the online conformal setting, propose several refinements in settings where additional structure is available, and demonstrate its effectiveness through experiments.

Related papers

Graph-Structured Feedback Multimodel Ensemble Online Conformal Prediction [14.188004615463742]
We propose a novel multi-model online conformal prediction algorithm.<n>It identifies a subset of effective models at each time step by collecting feedback from a bipartite graph.<n>A model is then selected from this subset to construct the prediction set, resulting in reduced computational complexity and smaller prediction sets.
arXiv Detail & Related papers (2025-06-26T00:06:11Z)
Adaptive Temperature Scaling with Conformal Prediction [47.51764759462074]
We propose to the best of our knowledge the first method for assigning calibrated probabilities to elements of a conformal prediction set.<n>Our approach frames this as an adaptive calibration problem, selecting an input-specific temperature parameter to match the desired coverage level.
arXiv Detail & Related papers (2025-05-21T12:18:15Z)
SConU: Selective Conformal Uncertainty in Large Language Models [59.25881667640868]
We propose a novel approach termed Selective Conformal Uncertainty (SConU)<n>We develop two conformal p-values that are instrumental in determining whether a given sample deviates from the uncertainty distribution of the calibration set at a specific manageable risk level.<n>Our approach not only facilitates rigorous management of miscoverage rates across both single-domain and interdisciplinary contexts, but also enhances the efficiency of predictions.
arXiv Detail & Related papers (2025-04-19T03:01:45Z)
Conformal Prediction Sets with Improved Conditional Coverage using Trust Scores [52.92618442300405]
It is impossible to achieve exact, distribution-free conditional coverage in finite samples.<n>We propose an alternative conformal prediction algorithm that targets coverage where it matters most.
arXiv Detail & Related papers (2025-01-17T12:01:56Z)
Multi-model Ensemble Conformal Prediction in Dynamic Environments [14.188004615463742]
We introduce a novel adaptive conformal prediction framework, where the model used for creating prediction sets is selected on the fly from multiple candidate models. The proposed algorithm is proven to achieve strongly adaptive regret over all intervals while maintaining valid coverage.
arXiv Detail & Related papers (2024-11-06T05:57:28Z)
Probabilistic Conformal Prediction with Approximate Conditional Validity [81.30551968980143]
We develop a new method for generating prediction sets that combines the flexibility of conformal methods with an estimate of the conditional distribution. Our method consistently outperforms existing approaches in terms of conditional coverage.
arXiv Detail & Related papers (2024-07-01T20:44:48Z)
Conformal Prediction for Ensembles: Improving Efficiency via Score-Based Aggregation [18.928543069018865]
Conformal prediction is one approach that avoids distributional assumptions.<n>Merging prediction regions directly, however, sacrifices structures present in the conformal scores that can further reduce conservatism.<n>We show that a novel framework can be efficiently leveraged in both classification and predict-then-optimize regression settings.
arXiv Detail & Related papers (2024-05-25T14:11:01Z)
Confidence on the Focal: Conformal Prediction with Selection-Conditional Coverage [6.010965256037659]
Conformal prediction builds marginally valid prediction intervals that cover the unknown outcome of a randomly drawn test point.<n>In practice, data-driven methods are often used to identify specific test unit(s) of interest.<n>This paper presents a general framework for constructing a prediction set with finite-sample exact coverage.
arXiv Detail & Related papers (2024-03-06T17:18:24Z)
Conformal Off-Policy Prediction in Contextual Bandits [54.67508891852636]
Conformal off-policy prediction can output reliable predictive intervals for the outcome under a new target policy. We provide theoretical finite-sample guarantees without making any additional assumptions beyond the standard contextual bandit setup.
arXiv Detail & Related papers (2022-06-09T10:39:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.