Distribution-free Conformal Prediction for Ordinal Classification

• URL: http://arxiv.org/abs/2404.16610v2
• Date: Sat, 02 Nov 2024 05:33:27 GMT
• Title: Distribution-free Conformal Prediction for Ordinal Classification
• Authors: Subhrasish Chakraborty, Chhavi Tyagi, Haiyan Qiao, Wenge Guo,
• Abstract summary: Ordinal classification is common in real applications where the target variable has natural ordering among the class labels. New conformal prediction methods are developed for constructing contiguous and non-contiguous prediction sets.
• Score: 0.0
• License:
• Abstract: Conformal prediction is a general distribution-free approach for constructing prediction sets combined with any machine learning algorithm that achieve valid marginal or conditional coverage in finite samples. Ordinal classification is common in real applications where the target variable has natural ordering among the class labels. In this paper, we discuss constructing distribution-free prediction sets for such ordinal classification problems by leveraging the ideas of conformal prediction and multiple testing with FWER control. Newer conformal prediction methods are developed for constructing contiguous and non-contiguous prediction sets based on marginal and conditional (class-specific) conformal $p$-values, respectively. Theoretically, we prove that the proposed methods respectively achieve satisfactory levels of marginal and class-specific conditional coverages. Through simulation study and real data analysis, these proposed methods show promising performance compared to the existing conformal method.

Related papers

• Optimal Transport-based Conformal Prediction [8.302146576157497]
  Conformal Prediction (CP) is a principled framework for uncertainty in blackbox learning models. We introduce a novel CP procedure handling prediction score functions through a lens. We then adapt our method for quantifying multi-output regression and multiclass classification.
  arXiv  Detail & Related papers  (2025-01-31T09:48:28Z)
• Conformal Prediction Sets with Improved Conditional Coverage using Trust Scores [52.92618442300405]
  It is impossible to achieve exact, distribution-free conditional coverage in finite samples. We propose an alternative conformal prediction algorithm that targets coverage where it matters most.
  arXiv  Detail & Related papers  (2025-01-17T12:01:56Z)
• Conformal Generative Modeling with Improved Sample Efficiency through Sequential Greedy Filtering [55.15192437680943]
  Generative models lack rigorous statistical guarantees for their outputs. We propose a sequential conformal prediction method producing prediction sets that satisfy a rigorous statistical guarantee. This guarantee states that with high probability, the prediction sets contain at least one admissible (or valid) example.
  arXiv  Detail & Related papers  (2024-10-02T15:26:52Z)
• Probabilistic Conformal Prediction with Approximate Conditional Validity [81.30551968980143]
  We develop a new method for generating prediction sets that combines the flexibility of conformal methods with an estimate of the conditional distribution. Our method consistently outperforms existing approaches in terms of conditional coverage.
  arXiv  Detail & Related papers  (2024-07-01T20:44:48Z)
• Class-Conditional Conformal Prediction with Many Classes [60.8189977620604]
  We propose a method called clustered conformal prediction that clusters together classes having "similar" conformal scores. We find that clustered conformal typically outperforms existing methods in terms of class-conditional coverage and set size metrics.
  arXiv  Detail & Related papers  (2023-06-15T17:59:02Z)
• Predictive Inference with Feature Conformal Prediction [80.77443423828315]
  We propose feature conformal prediction, which extends the scope of conformal prediction to semantic feature spaces. From a theoretical perspective, we demonstrate that feature conformal prediction provably outperforms regular conformal prediction under mild assumptions. Our approach could be combined with not only vanilla conformal prediction, but also other adaptive conformal prediction methods.
  arXiv  Detail & Related papers  (2022-10-01T02:57:37Z)
• Conformal prediction set for time-series [16.38369532102931]
  Uncertainty quantification is essential to studying complex machine learning methods. We develop Ensemble Regularized Adaptive Prediction Set (ERAPS) to construct prediction sets for time-series. We show valid marginal and conditional coverage by ERAPS, which also tends to yield smaller prediction sets than competing methods.
  arXiv  Detail & Related papers  (2022-06-15T23:48:53Z)
• Probabilistic Conformal Prediction Using Conditional Random Samples [73.26753677005331]
  PCP is a predictive inference algorithm that estimates a target variable by a discontinuous predictive set. It is efficient and compatible with either explicit or implicit conditional generative models.
  arXiv  Detail & Related papers  (2022-06-14T03:58:03Z)
• Practical Adversarial Multivalid Conformal Prediction [27.179891682629183]
  We give a generic conformal prediction method for sequential prediction. It achieves target empirical coverage guarantees against adversarially chosen data. It is computationally lightweight -- comparable to split conformal prediction.
  arXiv  Detail & Related papers  (2022-06-02T14:33:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.