Class conditional conformal prediction for multiple inputs by p-value aggregation

• URL: http://arxiv.org/abs/2507.07150v1
• Date: Wed, 09 Jul 2025 09:17:17 GMT
• Title: Class conditional conformal prediction for multiple inputs by p-value aggregation
• Authors: Jean-Baptiste Fermanian, Mohamed Hebiri, Joseph Salmon,
• Abstract summary: We introduce an innovative refinement to conformal prediction methods for classification tasks.<n>Our approach is motivated by applications in citizen science, where multiple images of the same plant or animal are captured by individuals.<n>We evaluate our method on simulated and real data, with a particular focus on Pl@ntNet, a prominent citizen science platform.
• Score: 11.198836025239963
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Conformal prediction methods are statistical tools designed to quantify uncertainty and generate predictive sets with guaranteed coverage probabilities. This work introduces an innovative refinement to these methods for classification tasks, specifically tailored for scenarios where multiple observations (multi-inputs) of a single instance are available at prediction time. Our approach is particularly motivated by applications in citizen science, where multiple images of the same plant or animal are captured by individuals. Our method integrates the information from each observation into conformal prediction, enabling a reduction in the size of the predicted label set while preserving the required class-conditional coverage guarantee. The approach is based on the aggregation of conformal p-values computed from each observation of a multi-input. By exploiting the exact distribution of these p-values, we propose a general aggregation framework using an abstract scoring function, encompassing many classical statistical tools. Knowledge of this distribution also enables refined versions of standard strategies, such as majority voting. We evaluate our method on simulated and real data, with a particular focus on Pl@ntNet, a prominent citizen science platform that facilitates the collection and identification of plant species through user-submitted images.

Related papers

• Multivariate Conformal Prediction using Optimal Transport [19.644272536912286]
  Conformal prediction (CP) quantifies the uncertainty of machine learning models.<n>We propose a principled framework for constructing conformal prediction sets in multidimensional settings.
  arXiv  Detail & Related papers  (2025-02-05T20:56:41Z)
• Optimal Transport-based Conformal Prediction [8.302146576157497]
  Conformal Prediction (CP) is a principled framework for uncertainty in blackbox learning models.<n>We introduce a novel CP procedure handling prediction score functions through a lens.<n>We then adapt our method for quantifying multi-output regression and multiclass classification.
  arXiv  Detail & Related papers  (2025-01-31T09:48:28Z)
• Conformal Prediction Sets with Improved Conditional Coverage using Trust Scores [52.92618442300405]
  It is impossible to achieve exact, distribution-free conditional coverage in finite samples.<n>We propose an alternative conformal prediction algorithm that targets coverage where it matters most.
  arXiv  Detail & Related papers  (2025-01-17T12:01:56Z)
• Probabilistic Conformal Prediction with Approximate Conditional Validity [81.30551968980143]
  We develop a new method for generating prediction sets that combines the flexibility of conformal methods with an estimate of the conditional distribution. Our method consistently outperforms existing approaches in terms of conditional coverage.
  arXiv  Detail & Related papers  (2024-07-01T20:44:48Z)
• Confidence on the Focal: Conformal Prediction with Selection-Conditional Coverage [6.010965256037659]
  Conformal prediction builds marginally valid prediction intervals that cover the unknown outcome of a randomly drawn test point.<n>In practice, data-driven methods are often used to identify specific test unit(s) of interest.<n>This paper presents a general framework for constructing a prediction set with finite-sample exact coverage.
  arXiv  Detail & Related papers  (2024-03-06T17:18:24Z)
• Class-Conditional Conformal Prediction with Many Classes [60.8189977620604]
  We propose a method called clustered conformal prediction that clusters together classes having "similar" conformal scores. We find that clustered conformal typically outperforms existing methods in terms of class-conditional coverage and set size metrics.
  arXiv  Detail & Related papers  (2023-06-15T17:59:02Z)
• Selection by Prediction with Conformal p-values [7.917044695538599]
  We study screening procedures that aim to select candidates whose unobserved outcomes exceed user-specified values. We develop a method that wraps around any prediction model to produce a subset of candidates while controlling the proportion of falsely selected units.
  arXiv  Detail & Related papers  (2022-10-04T06:34:49Z)
• Practical Adversarial Multivalid Conformal Prediction [27.179891682629183]
  We give a generic conformal prediction method for sequential prediction. It achieves target empirical coverage guarantees against adversarially chosen data. It is computationally lightweight -- comparable to split conformal prediction.
  arXiv  Detail & Related papers  (2022-06-02T14:33:00Z)
• Self-Certifying Classification by Linearized Deep Assignment [65.0100925582087]
  We propose a novel class of deep predictors for classifying metric data on graphs within PAC-Bayes risk certification paradigm. Building on the recent PAC-Bayes literature and data-dependent priors, this approach enables learning posterior distributions on the hypothesis space.
  arXiv  Detail & Related papers  (2022-01-26T19:59:14Z)
• Private Prediction Sets [72.75711776601973]
  Machine learning systems need reliable uncertainty quantification and protection of individuals' privacy. We present a framework that treats these two desiderata jointly. We evaluate the method on large-scale computer vision datasets.
  arXiv  Detail & Related papers  (2021-02-11T18:59:11Z)
• Certified Robustness to Label-Flipping Attacks via Randomized Smoothing [105.91827623768724]
  Machine learning algorithms are susceptible to data poisoning attacks. We present a unifying view of randomized smoothing over arbitrary functions. We propose a new strategy for building classifiers that are pointwise-certifiably robust to general data poisoning attacks.
  arXiv  Detail & Related papers  (2020-02-07T21:28:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.