Backward Conformal Prediction

• URL: http://arxiv.org/abs/2505.13732v1
• Date: Mon, 19 May 2025 21:08:14 GMT
• Title: Backward Conformal Prediction
• Authors: Etienne Gauthier, Francis Bach, Michael I. Jordan,
• Abstract summary: We introduce $textitBackward Conformal Prediction$, a method that guarantees conformal coverage while providing flexible control over the size of prediction sets.<n>Our approach defines a rule that constrains how prediction set sizes behave based on the observed data, and adapts the coverage level accordingly.<n>This approach is particularly useful in applications where large prediction sets are impractical such as medical diagnosis.
• Score: 49.1574468325115
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce $\textit{Backward Conformal Prediction}$, a method that guarantees conformal coverage while providing flexible control over the size of prediction sets. Unlike standard conformal prediction, which fixes the coverage level and allows the conformal set size to vary, our approach defines a rule that constrains how prediction set sizes behave based on the observed data, and adapts the coverage level accordingly. Our method builds on two key foundations: (i) recent results by Gauthier et al. [2025] on post-hoc validity using e-values, which ensure marginal coverage of the form $\mathbb{P}(Y_{\rm test} \in \hat C_n^{\tilde{\alpha}}(X_{\rm test})) \ge 1 - \mathbb{E}[\tilde{\alpha}]$ up to a first-order Taylor approximation for any data-dependent miscoverage $\tilde{\alpha}$, and (ii) a novel leave-one-out estimator $\hat{\alpha}^{\rm LOO}$ of the marginal miscoverage $\mathbb{E}[\tilde{\alpha}]$ based on the calibration set, ensuring that the theoretical guarantees remain computable in practice. This approach is particularly useful in applications where large prediction sets are impractical such as medical diagnosis. We provide theoretical results and empirical evidence supporting the validity of our method, demonstrating that it maintains computable coverage guarantees while ensuring interpretable, well-controlled prediction set sizes.

Related papers

• COIN: Uncertainty-Guarding Selective Question Answering for Foundation Models with Provable Risk Guarantees [51.5976496056012]
  COIN is an uncertainty-guarding selection framework that calibrates statistically valid thresholds to filter a single generated answer per question.<n>COIN estimates the empirical error rate on a calibration set and applies confidence interval methods to establish a high-probability upper bound on the true error rate.<n>We demonstrate COIN's robustness in risk control, strong test-time power in retaining admissible answers, and predictive efficiency under limited calibration data.
  arXiv  Detail & Related papers  (2025-06-25T07:04:49Z)
• Robust Conformal Prediction with a Single Binary Certificate [58.450154976190795]
  Conformal prediction (CP) converts any model's output to prediction sets with a guarantee to cover the true label with (adjustable) high probability.<n>We propose a robust conformal prediction that produces smaller sets even with significantly lower MC samples.
  arXiv  Detail & Related papers  (2025-03-07T08:41:53Z)
• Volume Optimality in Conformal Prediction with Structured Prediction Sets [22.923139209762788]
  Conformal Prediction is a widely studied technique to construct prediction sets of future observations.<n>We first prove an impossibility of volume optimality where any distribution-free method can only find a trivial solution.<n>We then introduce a new notion of volume optimality by restricting the prediction sets to belong to a set family.
  arXiv  Detail & Related papers  (2025-02-23T17:31:33Z)
• Conformal Prediction Sets with Improved Conditional Coverage using Trust Scores [52.92618442300405]
  It is impossible to achieve exact, distribution-free conditional coverage in finite samples.<n>We propose an alternative conformal prediction algorithm that targets coverage where it matters most.
  arXiv  Detail & Related papers  (2025-01-17T12:01:56Z)
• Probabilistic Conformal Prediction with Approximate Conditional Validity [81.30551968980143]
  We develop a new method for generating prediction sets that combines the flexibility of conformal methods with an estimate of the conditional distribution. Our method consistently outperforms existing approaches in terms of conditional coverage.
  arXiv  Detail & Related papers  (2024-07-01T20:44:48Z)
• Robust Conformal Prediction Using Privileged Information [17.886554223172517]
  We develop a method to generate prediction sets with a guaranteed coverage rate that is robust to corruptions in the training data.<n>Our approach builds on conformal prediction, a powerful framework to construct prediction sets that are valid under the i.i.d assumption.
  arXiv  Detail & Related papers  (2024-06-08T08:56:47Z)
• Provably Robust Conformal Prediction with Improved Efficiency [29.70455766394585]
  Conformal prediction is a powerful tool to generate uncertainty sets with guaranteed coverage. adversarial examples are able to manipulate conformal methods to construct prediction sets with invalid coverage rates. We propose two novel methods, Post-Training Transformation (PTT) and Robust Conformal Training (RCT), to effectively reduce prediction set size with little overhead.
  arXiv  Detail & Related papers  (2024-04-30T15:49:01Z)
• Post-selection Inference for Conformal Prediction: Trading off Coverage for Precision [0.0]
  Traditionally, conformal prediction inference requires a data-independent specification of miscoverage level. We develop simultaneous conformal inference to account for data-dependent miscoverage levels.
  arXiv  Detail & Related papers  (2023-04-12T20:56:43Z)
• Practical Adversarial Multivalid Conformal Prediction [27.179891682629183]
  We give a generic conformal prediction method for sequential prediction. It achieves target empirical coverage guarantees against adversarially chosen data. It is computationally lightweight -- comparable to split conformal prediction.
  arXiv  Detail & Related papers  (2022-06-02T14:33:00Z)
• Private Prediction Sets [72.75711776601973]
  Machine learning systems need reliable uncertainty quantification and protection of individuals' privacy. We present a framework that treats these two desiderata jointly. We evaluate the method on large-scale computer vision datasets.
  arXiv  Detail & Related papers  (2021-02-11T18:59:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.