On the Burden of Achieving Fairness in Conformal Prediction

• URL: http://arxiv.org/abs/2605.14260v2
• Date: Fri, 15 May 2026 16:13:21 GMT
• Title: On the Burden of Achieving Fairness in Conformal Prediction
• Abstract summary: We show that pooled calibration incurs irreducible group-wise coverage distortion at a scale set by cross-group quantile heterogeneity.<n>We quantify the cost of moving between policies which treat groups separately or pool them.
• Score: 8.720026908704371
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Conformal prediction is often calibrated with a single pooled threshold, but this can hide cross-group heterogeneity in score distributions and distort group-wise coverage. We study this phenomenon through the population score distributions underlying split conformal calibration. First, we derive a conservation law and lower bound showing that pooled calibration incurs irreducible group-wise coverage distortion at a scale set by cross-group quantile heterogeneity. Second, we demonstrate that the two leading fairness definitions for conformal prediction, Equalized Coverage and Equalized Set Size, are fundamentally in tension. Third, we quantify the cost of moving between policies which treat groups separately or pool them. Experiments on synthetic and real data confirm the same bidirectional trade-off after finite-sample calibration. Our results show that, for the policy families studied here, calibration choice does not remove cross-group heterogeneity; it determines whether the resulting distortion appears in the coverage or size dimension, providing a principled lens for analyzing fairness-oriented calibration choices in practice.

Related papers

• Conformal Bayes under Label Shift: Post-Hoc Calibration vs. In-Training Adaptation [0.6599344783327052]
  We study conformal Bayes under label shift from a unified perspective.<n>We identify two complementary approaches that restore nominal target-domain coverage.<n>In-training adaptation acts as a debiasing operator, reducing interval width at unchanged coverage.
  arXiv  Detail & Related papers  (2026-06-10T09:38:48Z)
• Fair regression under localized demographic parity constraints [9.082079838093371]
  We introduce a novel ($ell$, Z)-fair predictor, which imposes groupwise CDF constraints of the form F f |S=s (z m ) = $ell$ m for prescribed pairs.<n> Experiments on synthetic and real datasets illustrate an interpretable fairness-accuracy trade-off, enabling targeted corrections at decision-relevant quantiles or thresholds.
  arXiv  Detail & Related papers  (2026-03-26T09:21:05Z)
• Fix Representation (Optimally) Before Fairness: Finite-Sample Shrinkage Population Correction and the True Price of Fairness Under Subpopulation Shift [0.0]
  Machine learning practitioners frequently observe tension between predictive accuracy and group fairness constraints.<n>We show that both phenomena can be artifacts of training data that misrepresents subgroup proportions.
  arXiv  Detail & Related papers  (2026-02-05T14:32:05Z)
• Calibration Bands for Mean Estimates within the Exponential Dispersion Family [0.0]
  Testing for auto-calibration has only been considered recently in the literature.<n>We show that the obtained calibration bands allow us to construct various tests for calibration and auto-calibration.
  arXiv  Detail & Related papers  (2025-03-24T17:09:19Z)
• Conformal Prediction Sets with Improved Conditional Coverage using Trust Scores [52.92618442300405]
  It is impossible to achieve exact, distribution-free conditional coverage in finite samples.<n>We propose an alternative conformal prediction algorithm that targets coverage where it matters most.
  arXiv  Detail & Related papers  (2025-01-17T12:01:56Z)
• Theory on Score-Mismatched Diffusion Models and Zero-Shot Conditional Samplers [49.97755400231656]
  We present the first performance guarantee with explicit dimensional dependencies for general score-mismatched diffusion samplers.<n>We show that score mismatches result in an distributional bias between the target and sampling distributions, proportional to the accumulated mismatch between the target and training distributions.<n>This result can be directly applied to zero-shot conditional samplers for any conditional model, irrespective of measurement noise.
  arXiv  Detail & Related papers  (2024-10-17T16:42:12Z)
• Adaptive Uncertainty Quantification for Generative AI [0.0]
  Mirroring split-conformal inference, we design a wrapper around a black-box algorithm which calibrates conformity scores.<n> adaptive partitioning is achieved by fitting a robust regression tree to the conformity scores on the calibration set.<n>Unlike traditional split-conformal inference, adaptive splitting and within-group calibration yields adaptive bands which can stretch and shrink locally.
  arXiv  Detail & Related papers  (2024-08-16T19:37:33Z)
• Equal Opportunity of Coverage in Fair Regression [50.76908018786335]
  We study fair machine learning (ML) under predictive uncertainty to enable reliable and trustworthy decision-making. We propose Equal Opportunity of Coverage (EOC) that aims to achieve two properties: (1) coverage rates for different groups with similar outcomes are close, and (2) the coverage rate for the entire population remains at a predetermined level.
  arXiv  Detail & Related papers  (2023-11-03T21:19:59Z)
• Correcting Underrepresentation and Intersectional Bias for Classification [49.1574468325115]
  We consider the problem of learning from data corrupted by underrepresentation bias. We show that with a small amount of unbiased data, we can efficiently estimate the group-wise drop-out rates. We show that our algorithm permits efficient learning for model classes of finite VC dimension.
  arXiv  Detail & Related papers  (2023-06-19T18:25:44Z)
• Distribution-free uncertainty quantification for classification under label shift [105.27463615756733]
  We focus on uncertainty quantification (UQ) for classification problems via two avenues. We first argue that label shift hurts UQ, by showing degradation in coverage and calibration. We examine these techniques theoretically in a distribution-free framework and demonstrate their excellent practical performance.
  arXiv  Detail & Related papers  (2021-03-04T20:51:03Z)
• Calibration of Neural Networks using Splines [51.42640515410253]
  Measuring calibration error amounts to comparing two empirical distributions. We introduce a binning-free calibration measure inspired by the classical Kolmogorov-Smirnov (KS) statistical test. Our method consistently outperforms existing methods on KS error as well as other commonly used calibration measures.
  arXiv  Detail & Related papers  (2020-06-23T07:18:05Z)
• Distribution-free binary classification: prediction sets, confidence intervals and calibration [106.50279469344937]
  We study three notions of uncertainty quantification -- calibration, confidence intervals and prediction sets -- for binary classification in the distribution-free setting. We derive confidence intervals for binned probabilities for both fixed-width and uniform-mass binning. As a consequence of our 'tripod' theorems, these confidence intervals for binned probabilities lead to distribution-free calibration.
  arXiv  Detail & Related papers  (2020-06-18T14:17:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.