Reaching the Tail: Calibration Diversity Drives Conformal Coverage under Data Scarcity

• URL: http://arxiv.org/abs/2608.21591v1
• Date: Fri, 21 Aug 2026 19:51:08 GMT
• Title: Reaching the Tail: Calibration Diversity Drives Conformal Coverage under Data Scarcity
• Abstract summary: Multi-horizon rare-event forecasting is hard under long macroeconomic series' data constraints.<n>A controlled ablation shows an apparent rare-event threshold for Adaptive Conformal Inference instead reflects calibration-set size.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-horizon rare-event forecasting is hard under long macroeconomic series' data constraints: labeled events are scarce, and standard uncertainty quantification assumes an exchangeability that autocorrelation violates. A controlled ablation shows an apparent rare-event threshold for Adaptive Conformal Inference instead reflects calibration-set size. Across 200 random calibration sets, support width of the nonconformity-score distribution explains up to 85% of coverage variance versus 2% for rare-event count; the same, not the same magnitude, replicates across synthetic conditions and five countries (five-country Spearman $ρ$ 0.45-0.66 vs. 0.02-0.23). A diversity-maximizing selector built on this is the only strategy tested that improves long-horizon coverage (67.8% to 81.4% at six months); Mondrian, shift-robust, and extreme-value alternatives fail to close it. Mondrian even worsens coverage under oracle labels. A compact proposition explains why: coverage deficit reflects how closely the calibration set's upper quantile reaches the test distribution's. Diversity is necessary, not sufficient. Demonstrated on a two-stage U.S. recession-forecasting framework with RegressorChain, whether six-month coverage reaches 90% under honest scoring remains open, a question this paper quantifies rather than resolves.

Related papers

• Conformal Calibration for Multi-Modal Regression with Missing Modalities [0.0]
  Prediction intervals for multi-modal regression are difficult to calibrate when sources disagree or one is missing.<n>We address this through a modality-aware conformal calibration layer.<n>The layer trains or reuses one predictor per modality, computes a disagreement score from their predictions, and uses that score in split conformal calibration.
  arXiv  Detail & Related papers  (2026-08-07T22:38:35Z)
• Distribution-Free Conformal Prediction for Steel Fatigue Strength: Marginal Validity Is Not Enough [0.0]
  Predicting fatigue failure in steel components experimentally is costly because it requires testing across multiple compositions and processing conditions.<n>Studies using the NIMS MatNavi steel fatigue dataset often report high point-prediction accuracy but rely on aggregate error metrics.<n>This paper is the first to apply conformal prediction to steel fatigue strength.
  arXiv  Detail & Related papers  (2026-08-05T18:13:28Z)
• Post-Hoc Trajectory-Risk Certification for Modular LLM-Based Security Agents [1.4915002678801434]
  We show that Bonferroni allocation is distribution-free but conservative under correlated errors.<n> coarse-to-fine label selection can create near-perfect measured correlation without learned dependence.<n>A modular certificate using per-stage certificates and a pairwise overlap bound yields a positive average gain of 0.6%.
  arXiv  Detail & Related papers  (2026-08-04T23:48:36Z)
• A Quiet Failure in Calibrated Virtual Screening: Marginal Conformal Prediction Under-Covers the Minority Class, and a Class-Conditional Fix Recovers It [0.0]
  Conformal prediction is being adopted in drug discovery to put an honest number on model reliability.<n>We show this guarantee can be dangerous on imbalanced datasets.<n>Our contribution is demonstrating on real chemistry how severe and invisible this known conformal-theory gap becomes under imbalance.
  arXiv  Detail & Related papers  (2026-07-07T04:31:56Z)
• Decoupled Conformal Optimisation: Efficient Prediction Sets via Independent Tuning and Calibration [20.21455697379946]
  We propose Decoupled Conformal optimisation (DCO) as a train-tune-calibrate design principle.<n>DCO uses an independent tuning split for efficiency-oriented structural selection and a fresh calibration split for the final conformal quantile.<n>We show that DCO tracks the nominal coverage level closely while often reducing average prediction-set size or interval width relative to PAC-style calibration.
  arXiv  Detail & Related papers  (2026-05-18T13:10:21Z)
• Conditional Coverage Diagnostics for Conformal Prediction [47.93989136542648]
  We show that conditional coverage estimation can be a classification problem.<n>We call the resulting family of metrics excess risk of the target coverage (ERT)<n>We release an open-source package for ERT as well as previous conditional coverage metrics.
  arXiv  Detail & Related papers  (2025-12-12T18:47:39Z)
• COIN: Uncertainty-Guarding Selective Question Answering for Foundation Models with Provable Risk Guarantees [51.5976496056012]
  COIN is an uncertainty-guarding selection framework that calibrates statistically valid thresholds to filter a single generated answer per question.<n>COIN estimates the empirical error rate on a calibration set and applies confidence interval methods to establish a high-probability upper bound on the true error rate.<n>We demonstrate COIN's robustness in risk control, strong test-time power in retaining admissible answers, and predictive efficiency under limited calibration data.
  arXiv  Detail & Related papers  (2025-06-25T07:04:49Z)
• Conformal Prediction Sets with Improved Conditional Coverage using Trust Scores [52.92618442300405]
  It is impossible to achieve exact, distribution-free conditional coverage in finite samples.<n>We propose an alternative conformal prediction algorithm that targets coverage where it matters most.
  arXiv  Detail & Related papers  (2025-01-17T12:01:56Z)
• Equal Opportunity of Coverage in Fair Regression [50.76908018786335]
  We study fair machine learning (ML) under predictive uncertainty to enable reliable and trustworthy decision-making. We propose Equal Opportunity of Coverage (EOC) that aims to achieve two properties: (1) coverage rates for different groups with similar outcomes are close, and (2) the coverage rate for the entire population remains at a predetermined level.
  arXiv  Detail & Related papers  (2023-11-03T21:19:59Z)
• Integrating Uncertainty Awareness into Conformalized Quantile Regression [12.875863572064986]
  We propose a new variant of the Conformalized Quantile Regression (CQR) methodology to adjust quantile regressors differentially across the feature space. Compared to CQR, our methods enjoy the same distribution-free theoretical coverage guarantees, while demonstrating stronger conditional coverage properties in simulated settings and real-world data sets alike.
  arXiv  Detail & Related papers  (2023-06-14T18:28:53Z)
• Distribution-free uncertainty quantification for classification under label shift [105.27463615756733]
  We focus on uncertainty quantification (UQ) for classification problems via two avenues. We first argue that label shift hurts UQ, by showing degradation in coverage and calibration. We examine these techniques theoretically in a distribution-free framework and demonstrate their excellent practical performance.
  arXiv  Detail & Related papers  (2021-03-04T20:51:03Z)
• A Unified Joint Maximum Mean Discrepancy for Domain Adaptation [73.44809425486767]
  This paper theoretically derives a unified form of JMMD that is easy to optimize. From the revealed unified JMMD, we illustrate that JMMD degrades the feature-label dependence that benefits to classification. We propose a novel MMD matrix to promote the dependence, and devise a novel label kernel that is robust to label distribution shift.
  arXiv  Detail & Related papers  (2021-01-25T09:46:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.