Split Localized Conformal Prediction
- URL: http://arxiv.org/abs/2206.13092v1
- Date: Mon, 27 Jun 2022 07:53:38 GMT
- Title: Split Localized Conformal Prediction
- Authors: Xing Han, Ziyang Tang, Joydeep Ghosh, Qiang Liu
- Abstract summary: We propose a modified non-conformity score by leveraging local approximation of the conditional distribution.
The modified score inherits the spirit of split conformal methods, which is simple and efficient compared with full conformal methods.
- Score: 20.44976410408424
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Conformal prediction is a simple and powerful tool that can quantify
uncertainty without any distributional assumptions. However, existing methods
can only provide an average coverage guarantee, which is not ideal compared to
the stronger conditional coverage guarantee. Although achieving exact
conditional coverage is proven to be impossible, approximating conditional
coverage is still an important research direction. In this paper, we propose a
modified non-conformity score by leveraging local approximation of the
conditional distribution. The modified score inherits the spirit of split
conformal methods, which is simple and efficient compared with full conformal
methods but better approximates conditional coverage guarantee. Empirical
results on various datasets, including a high dimension age regression on
image, demonstrate that our method provides tighter intervals compared to
existing methods.
Related papers
- Conditionally valid Probabilistic Conformal Prediction [57.80927226809277]
We develop a new method for creating prediction sets that combines the flexibility of conformal methods with an estimate of the conditional distribution.
We demonstrate the effectiveness of our approach through extensive simulations, showing that it outperforms existing methods in terms of conditional coverage.
arXiv Detail & Related papers (2024-07-01T20:44:48Z) - Efficient Conformal Prediction under Data Heterogeneity [79.35418041861327]
Conformal Prediction (CP) stands out as a robust framework for uncertainty quantification.
Existing approaches for tackling non-exchangeability lead to methods that are not computable beyond the simplest examples.
This work introduces a new efficient approach to CP that produces provably valid confidence sets for fairly general non-exchangeable data distributions.
arXiv Detail & Related papers (2023-12-25T20:02:51Z) - Equal Opportunity of Coverage in Fair Regression [50.76908018786335]
We study fair machine learning (ML) under predictive uncertainty to enable reliable and trustworthy decision-making.
We propose Equal Opportunity of Coverage (EOC) that aims to achieve two properties: (1) coverage rates for different groups with similar outcomes are close, and (2) the coverage rate for the entire population remains at a predetermined level.
arXiv Detail & Related papers (2023-11-03T21:19:59Z) - Class-Conditional Conformal Prediction with Many Classes [60.8189977620604]
We propose a method called clustered conformal prediction that clusters together classes having "similar" conformal scores.
We find that clustered conformal typically outperforms existing methods in terms of class-conditional coverage and set size metrics.
arXiv Detail & Related papers (2023-06-15T17:59:02Z) - Adaptive Conformal Regression with Jackknife+ Rescaled Scores [7.176758110912026]
Conformal regression provides prediction intervals with global coverage guarantees, but often fails to capture local error distributions.
We address this with a new adaptive method based on rescaling conformal scores with an estimate of local score distribution.
Our approach ensures formal global coverage guarantees and is supported by new theoretical results on local coverage, including an a posteriori bound on any calibration score.
arXiv Detail & Related papers (2023-05-31T14:32:26Z) - Conformal Off-Policy Prediction in Contextual Bandits [54.67508891852636]
Conformal off-policy prediction can output reliable predictive intervals for the outcome under a new target policy.
We provide theoretical finite-sample guarantees without making any additional assumptions beyond the standard contextual bandit setup.
arXiv Detail & Related papers (2022-06-09T10:39:33Z) - Practical Adversarial Multivalid Conformal Prediction [27.179891682629183]
We give a generic conformal prediction method for sequential prediction.
It achieves target empirical coverage guarantees against adversarially chosen data.
It is computationally lightweight -- comparable to split conformal prediction.
arXiv Detail & Related papers (2022-06-02T14:33:00Z) - Uncertainty Sets for Image Classifiers using Conformal Prediction [112.54626392838163]
We present an algorithm that modifies any classifier to output a predictive set containing the true label with a user-specified probability, such as 90%.
The algorithm is simple and fast like Platt scaling, but provides a formal finite-sample coverage guarantee for every model and dataset.
Our method modifies an existing conformal prediction algorithm to give more stable predictive sets by regularizing the small scores of unlikely classes after Platt scaling.
arXiv Detail & Related papers (2020-09-29T17:58:04Z) - Classification with Valid and Adaptive Coverage [11.680355561258427]
Conformal inference, cross-validation+, and the jackknife+ are hold-out methods that can be combined with virtually any machine learning algorithm.
We develop specialized versions of these techniques for categorical and unordered response labels.
arXiv Detail & Related papers (2020-06-03T21:42:04Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.