Conformal Prediction Meets Long-tail Classification

• URL: http://arxiv.org/abs/2508.11345v1
• Date: Fri, 15 Aug 2025 09:14:46 GMT
• Title: Conformal Prediction Meets Long-tail Classification
• Abstract summary: Conformal Prediction (CP) methods often exhibit imbalanced coverage across classes under long-tail label distributions.<n>We propose the Tail-Aware Conformal Prediction (TACP) method to mitigate the under coverage of the tail classes.
• Score: 5.3663546125491735
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Conformal Prediction (CP) is a popular method for uncertainty quantification that converts a pretrained model's point prediction into a prediction set, with the set size reflecting the model's confidence. Although existing CP methods are guaranteed to achieve marginal coverage, they often exhibit imbalanced coverage across classes under long-tail label distributions, tending to over cover the head classes at the expense of under covering the remaining tail classes. This under coverage is particularly concerning, as it undermines the reliability of the prediction sets for minority classes, even with coverage ensured on average. In this paper, we propose the Tail-Aware Conformal Prediction (TACP) method to mitigate the under coverage of the tail classes by utilizing the long-tail structure and narrowing the head-tail coverage gap. Theoretical analysis shows that it consistently achieves a smaller head-tail coverage gap than standard methods. To further improve coverage balance across all classes, we introduce an extension of TACP: soft TACP (sTACP) via a reweighting mechanism. The proposed framework can be combined with various non-conformity scores, and experiments on multiple long-tail benchmark datasets demonstrate the effectiveness of our methods.

Related papers

• Audited Conformal Prediction for Classification under Unknown Distribution Shift [18.25994880093428]
  We propose Audited Conformal Prediction to train an auxiliary audit model identifying inputs where the legacy model is likely to fail.<n>ACP produces prediction sets that guarantee marginal coverage while achieving substantially higher conditional coverage in practice than existing approaches.
  arXiv  Detail & Related papers  (2026-06-12T19:28:40Z)
• Leverage-Weighted Conformal Prediction [1.3537117504260623]
  Split conformal prediction provides distribution-free prediction intervals with finite-sample marginal coverage.<n>We propose Leverage-Weighted Conformal Prediction, which weights nonconformity scores by a function of the statistical leverage.
  arXiv  Detail & Related papers  (2026-02-13T07:49:39Z)
• ST-BCP: Tightening Coverage Bound for Backward Conformal Prediction via Non-Conformity Score Transformation [18.272247805086284]
  Conformal Prediction (CP) provides a statistical framework for uncertainty quantification that constructs prediction sets with coverage guarantees.<n>BCP inverts this paradigm by enforcing a predefined upper bound on set size and estimating the resulting coverage guarantee.<n>We introduce ST-BCP, a novel method that introduces a data-dependent transformation of nonconformity scores to narrow the coverage gap.
  arXiv  Detail & Related papers  (2026-02-02T07:18:35Z)
• Distribution-informed Online Conformal Prediction [53.674678995825666]
  We propose Conformal Optimistic Prediction (COP), an online conformal prediction algorithm incorporating underlying data pattern into the update rule.<n>COP produces tighter prediction sets when predictable pattern exists, while retaining valid coverage guarantees even when estimates are inaccurate.<n>We prove that COP can achieve valid coverage and construct shorter prediction intervals than other baselines.
  arXiv  Detail & Related papers  (2025-12-08T17:51:49Z)
• Provably Minimum-Length Conformal Prediction Sets for Ordinal Classification [17.822517655243427]
  We propose an ordinal-CP method that is model-agnostic and provides instance-level optimal prediction intervals.<n>Experiments on four benchmark datasets from diverse domains are conducted to demonstrate the significantly improved predictive efficiency.
  arXiv  Detail & Related papers  (2025-11-20T23:00:15Z)
• Conformal Prediction Sets with Improved Conditional Coverage using Trust Scores [52.92618442300405]
  It is impossible to achieve exact, distribution-free conditional coverage in finite samples.<n>We propose an alternative conformal prediction algorithm that targets coverage where it matters most.
  arXiv  Detail & Related papers  (2025-01-17T12:01:56Z)
• Conformal Depression Prediction [2.097941594997818]
  Conformal depression prediction (CDP) is a depression prediction method with uncertainty quantification based on conformal prediction (CP) CDP is a plug-and-play module that requires neither model retraining nor an assumption about the depression data distribution. We propose CDP-ACC, an improved conformal prediction with approximate conditional coverage.
  arXiv  Detail & Related papers  (2024-05-29T03:08:30Z)
• Conformal Prediction with Learned Features [22.733758606168873]
  We propose Partition Learning Conformal Prediction (PLCP) to improve conditional validity of prediction sets. We implement PLCP efficiently with gradient alternating descent, utilizing off-the-shelf machine learning models. Our experimental results over four real-world and synthetic datasets show the superior performance of PLCP.
  arXiv  Detail & Related papers  (2024-04-26T15:43:06Z)
• Equal Opportunity of Coverage in Fair Regression [50.76908018786335]
  We study fair machine learning (ML) under predictive uncertainty to enable reliable and trustworthy decision-making. We propose Equal Opportunity of Coverage (EOC) that aims to achieve two properties: (1) coverage rates for different groups with similar outcomes are close, and (2) the coverage rate for the entire population remains at a predetermined level.
  arXiv  Detail & Related papers  (2023-11-03T21:19:59Z)
• Conformal Prediction for Federated Uncertainty Quantification Under Label Shift [57.54977668978613]
  Federated Learning (FL) is a machine learning framework where many clients collaboratively train models. We develop a new conformal prediction method based on quantile regression and take into account privacy constraints.
  arXiv  Detail & Related papers  (2023-06-08T11:54:58Z)
• Selective Classification via One-Sided Prediction [54.05407231648068]
  One-sided prediction (OSP) based relaxation yields an SC scheme that attains near-optimal coverage in the practically relevant high target accuracy regime. We theoretically derive bounds generalization for SC and OSP, and empirically we show that our scheme strongly outperforms state of the art methods in coverage at small error levels.
  arXiv  Detail & Related papers  (2020-10-15T16:14:27Z)
• Learning, compression, and leakage: Minimising classification error via meta-universal compression principles [87.054014983402]
  A promising group of compression techniques for learning scenarios is normalised maximum likelihood (NML) coding. Here we consider a NML-based decision strategy for supervised classification problems, and show that it attains PAC learning when applied to a wide variety of models. We show that the misclassification rate of our method is upper bounded by the maximal leakage, a recently proposed metric to quantify the potential of data leakage in privacy-sensitive scenarios.
  arXiv  Detail & Related papers  (2020-10-14T20:03:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.