When Is a Conformal Guarantee Fair? Auditing Silent Subgroup Under-Coverage in Alzheimer's Disease Longitudinal Prediction

• URL: http://arxiv.org/abs/2608.04254v1
• Date: Tue, 04 Aug 2026 22:20:44 GMT
• Title: When Is a Conformal Guarantee Fair? Auditing Silent Subgroup Under-Coverage in Alzheimer's Disease Longitudinal Prediction
• Abstract summary: We introduce a general mechanism-driven framework for auditing and repairing subgroup under-coverage.<n>We find that population-level bands under-cover high-risk subgroups in 57 of 68 audited combinations, despite achieving nominal marginal coverage.<n>We pair each mechanism with a corresponding conformal correction: cross-conformal pooling for rarity, per-subgroup calibration for tail-heaviness, and a coverage-safe marginal floor.
• Score: 3.8892887527569857
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Longitudinal prediction of Alzheimer's disease biomarkers increasingly informs clinical decisions, and a forecast is only useful if it also reports how much to trust it. Conformal prediction supplies this by wrapping any forecaster in a prediction band with a finite-sample coverage guarantee under exchangeability. However, standard population-level conformal prediction guarantees only marginal coverage and may mask substantial under-coverage within clinically important subgroups. We introduce a general mechanism-driven framework for auditing and repairing such subgroup under-coverage. Across two cohorts (ADNI, OASIS-3), two base forecasters, and nine attributes spanning genetic risk, demographics, and clinical severity, we find that population-level bands under-cover high-risk subgroups in 57 of 68 audited combinations, despite achieving nominal marginal coverage. We trace these failures to two mechanisms: (A) \emph{rarity}, where a group-conditional band calibrated on only $n$ patients covers at most $k/(n+1)$; and (B) \emph{tail-heaviness}, where a population-wide band is too narrow for a heavy-tailed subgroup and additional data cannot close the gap. Under-coverage falls disproportionately on patients with high genetic risk and disease severity (6.1 pp mean deficit, 95\% CI [3.3, 8.9]), while demographic groups remain at the target level on average (0.0 pp, CI [$-1.9$, 1.7]). We pair each mechanism with a corresponding conformal correction: cross-conformal pooling for rarity, per-subgroup calibration for tail-heaviness, and a coverage-safe marginal floor when both arise. Together, these corrections restore target coverage for nearly every high-risk subgroup across both cohorts and forecasters.

Related papers

• When Calibration Fails the Vulnerable Hospital: Federated Conformal Risk Control via Risk-Curve Shrinkage [0.0]
  Conformal risk control (CRC) provides distribution-free guarantees on segmentation quality.<n>In federated deployments, the standard approach pools calibration scores across sites into a single threshold.<n>We show that this naive pooled CRC protects the average hospital but violates coverage at 40% of individual institutions.
  arXiv  Detail & Related papers  (2026-06-18T11:39:44Z)
• Conformal Risk Prediction for Non-Alcoholic Fatty Liver Disease Using Gradient Boosting with Distribution-Free Coverages [2.538209532048867]
  Non-alcoholic fatty liver disease (NAFLD) affects roughly 25% of global adults.<n>We present Method, a machine-learning framework for NAFLD risk prediction.
  arXiv  Detail & Related papers  (2026-05-31T14:23:21Z)
• Socio-Conformal Calibration in Complex Survey Data: Marginal Validity Is Not Enough for Subgroup Reliability [1.089614199781423]
  We study ordinal conformal prediction for five-level AI-attitude forecasting on the Pew American Trends Panel.<n>Standard conformal achieves nominal marginal coverage for all four base predictors but leaves weighted subgroup gaps of 13 percentage points.<n>For the strongest predictor (XGBoost), Mondrian worsens the fairness-efficiency trade-off.<n>A regularized comparator that shrinks group thresholds toward the global quantile mitigates this instability.
  arXiv  Detail & Related papers  (2026-05-07T01:10:48Z)
• Conditional Coverage Diagnostics for Conformal Prediction [47.93989136542648]
  We show that conditional coverage estimation can be a classification problem.<n>We call the resulting family of metrics excess risk of the target coverage (ERT)<n>We release an open-source package for ERT as well as previous conditional coverage metrics.
  arXiv  Detail & Related papers  (2025-12-12T18:47:39Z)
• Developing Fairness-Aware Task Decomposition to Improve Equity in Post-Spinal Fusion Complication Prediction [3.860970992977915]
  We propose a fairness-aware multitask learning framework for postoperative complication prediction.<n> FAIR-MTL employs a data-driven subgroup inference mechanism.<n>It achieves an AUC of 0.86 and an accuracy of 75%, outperforming single-task baselines while substantially reducing bias.
  arXiv  Detail & Related papers  (2025-11-29T19:06:07Z)
• Hybrid Adaptive Conformal Offline Reinforcement Learning for Fair Population Health Management [1.5635627702544692]
  Population health management programs for Medicaid populations coordinate longitudinal outreach and services.<n>We present a Hybrid Adaptive Conformal Offline Reinforcement Learning framework that separates risk calibration from preference optimization to generate conservative action recommendations.
  arXiv  Detail & Related papers  (2025-09-11T18:09:28Z)
• Conformal Prediction Sets with Improved Conditional Coverage using Trust Scores [52.92618442300405]
  It is impossible to achieve exact, distribution-free conditional coverage in finite samples.<n>We propose an alternative conformal prediction algorithm that targets coverage where it matters most.
  arXiv  Detail & Related papers  (2025-01-17T12:01:56Z)
• Using Pre-training and Interaction Modeling for ancestry-specific disease prediction in UK Biobank [69.90493129893112]
  Recent genome-wide association studies (GWAS) have uncovered the genetic basis of complex traits, but show an under-representation of non-European descent individuals. Here, we assess whether we can improve disease prediction across diverse ancestries using multiomic data.
  arXiv  Detail & Related papers  (2024-04-26T16:39:50Z)
• How does promoting the minority fraction affect generalization? A theoretical study of the one-hidden-layer neural network on group imbalance [64.1656365676171]
  Group imbalance has been a known problem in empirical risk minimization. This paper quantifies the impact of individual groups on the sample complexity, the convergence rate, and the average and group-level testing performance.
  arXiv  Detail & Related papers  (2024-03-12T04:38:05Z)
• Equal Opportunity of Coverage in Fair Regression [50.76908018786335]
  We study fair machine learning (ML) under predictive uncertainty to enable reliable and trustworthy decision-making. We propose Equal Opportunity of Coverage (EOC) that aims to achieve two properties: (1) coverage rates for different groups with similar outcomes are close, and (2) the coverage rate for the entire population remains at a predetermined level.
  arXiv  Detail & Related papers  (2023-11-03T21:19:59Z)
• Learning to diagnose cirrhosis from radiological and histological labels with joint self and weakly-supervised pretraining strategies [62.840338941861134]
  We propose to leverage transfer learning from large datasets annotated by radiologists, to predict the histological score available on a small annex dataset. We compare different pretraining methods, namely weakly-supervised and self-supervised ones, to improve the prediction of the cirrhosis. This method outperforms the baseline classification of the METAVIR score, reaching an AUC of 0.84 and a balanced accuracy of 0.75.
  arXiv  Detail & Related papers  (2023-02-16T17:06:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.