Conformal Bayes under Label Shift: Post-Hoc Calibration vs. In-Training Adaptation

• URL: http://arxiv.org/abs/2606.11865v1
• Date: Wed, 10 Jun 2026 09:38:48 GMT
• Title: Conformal Bayes under Label Shift: Post-Hoc Calibration vs. In-Training Adaptation
• Abstract summary: We study conformal Bayes under label shift from a unified perspective.<n>We identify two complementary approaches that restore nominal target-domain coverage.<n>In-training adaptation acts as a debiasing operator, reducing interval width at unchanged coverage.
• Score: 0.6599344783327052
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Conformal Bayes combines Bayesian posterior predictives with conformal calibration to produce prediction sets that are both statistically valid and geometrically efficient. We study conformal Bayes under label shift from a unified perspective, identifying two complementary approaches that restore nominal target-domain coverage through importance-weighted conformal calibration but operate through independent mechanisms. \emph{Post-hoc calibration} tilts the posterior predictive toward the target domain and corrects the conformal threshold via an importance-weighted quantile, leaving the parameter posterior unchanged. \emph{In-training adaptation} tilts the parameter posterior itself to the target domain, producing a corrected predictive whose highest predictive density region serves as the highest predictive density (HPD) based prediction set under the fitted target predictive; efficiency is model-dependent and does not imply finite-sample conditional optimality. Two controlled experiments show that in an unbiased training regime both strategies achieve valid coverage equally, while in a lead-optimization regime in-training adaptation acts as a debiasing operator, reducing interval width at unchanged coverage.

Related papers

• Isotonic Conformal Prediction [1.0499611180329804]
  We consider two objectives for reliable uncertainty quantification: self-calibration and prediction-conditional validity.<n>Self-Calibrating Conformal Prediction attains both objectives exactly in finite samples, but requires refitting its calibrator for every candidate outcome.<n>We propose Isotonic Conformal Prediction (ICP), a framework that decouples calibration from prediction-set construction.
  arXiv  Detail & Related papers  (2026-07-18T07:11:05Z)
• Set-Preserving Calibration from Conformal P-Values to E-Values [6.673032375204486]
  We propose a novel P2E calibrator that converts conformal p-values into e-values without altering the prediction set induced by the original conformal p-value.<n>This e-value formulation enables principled use of recent advances in e-value merging and randomization.
  arXiv  Detail & Related papers  (2026-06-02T13:06:01Z)
• Skew-adaptive conformal prediction [1.0312968200748116]
  We develop a skew-adaptive extension of split conformal prediction for regression.<n>The resulting procedure preserves the finite-sample marginal validity of split conformal prediction under exchangeability.<n>We also develop a calibration-sample-based estimator for comparing the expected relative future width of the skew-adaptive and classical scaled-score intervals.
  arXiv  Detail & Related papers  (2026-05-15T16:26:41Z)
• Multi-Variable Conformal Prediction: Optimizing Prediction Sets without Data Splitting [16.714269073574428]
  We introduce multivariable conformal prediction (MCP), a framework that extends conformal prediction functions with multiple simultaneous variables.<n>We demonstrate that RemMCP and RelMCP consistently meet the target coverage problem with prediction set sizes smaller than or comparable to the target coverage set sizes.
  arXiv  Detail & Related papers  (2026-05-12T16:19:51Z)
• Coverage Guarantees for Pseudo-Calibrated Conformal Prediction under Distribution Shift [1.5861469511290378]
  Conformal prediction offers marginal coverage guarantees if the data distribution shifts.<n>We analyze the use of pseudo-calibration as a tool to counter this performance loss.<n>We propose a source-tuned pseudo-calibration algorithm that interpolates between hard pseudo-labels and randomized labels.
  arXiv  Detail & Related papers  (2026-02-16T16:48:39Z)
• UAT-LITE: Inference-Time Uncertainty-Aware Attention for Pretrained Transformers [11.741258610945259]
  UAT-LITE is an inference-time framework that makes self-attention uncertainty-aware.<n>It reduces Expected Error by approximately 20% on average relative to a fine-tuned BERT-base baseline.<n>It improves selective prediction and robustness under distribution shift.
  arXiv  Detail & Related papers  (2026-02-03T00:51:26Z)
• Distribution-informed Online Conformal Prediction [53.674678995825666]
  We propose Conformal Optimistic Prediction (COP), an online conformal prediction algorithm incorporating underlying data pattern into the update rule.<n>COP produces tighter prediction sets when predictable pattern exists, while retaining valid coverage guarantees even when estimates are inaccurate.<n>We prove that COP can achieve valid coverage and construct shorter prediction intervals than other baselines.
  arXiv  Detail & Related papers  (2025-12-08T17:51:49Z)
• Rectifying Conformity Scores for Better Conditional Coverage [75.73184036344908]
  We present a new method for generating confidence sets within the split conformal prediction framework.<n>Our method performs a trainable transformation of any given conformity score to improve conditional coverage while ensuring exact marginal coverage.
  arXiv  Detail & Related papers  (2025-02-22T19:54:14Z)
• Conformal Prediction Sets with Improved Conditional Coverage using Trust Scores [52.92618442300405]
  It is impossible to achieve exact, distribution-free conditional coverage in finite samples.<n>We propose an alternative conformal prediction algorithm that targets coverage where it matters most.
  arXiv  Detail & Related papers  (2025-01-17T12:01:56Z)
• Provably Reliable Conformal Prediction Sets in the Presence of Data Poisoning [53.42244686183879]
  Conformal prediction provides model-agnostic and distribution-free uncertainty quantification.<n>Yet, conformal prediction is not reliable under poisoning attacks where adversaries manipulate both training and calibration data.<n>We propose reliable prediction sets (RPS): the first efficient method for constructing conformal prediction sets with provable reliability guarantees under poisoning.
  arXiv  Detail & Related papers  (2024-10-13T15:37:11Z)
• Improving Adaptive Conformal Prediction Using Self-Supervised Learning [72.2614468437919]
  We train an auxiliary model with a self-supervised pretext task on top of an existing predictive model and use the self-supervised error as an additional feature to estimate nonconformity scores. We empirically demonstrate the benefit of the additional information using both synthetic and real data on the efficiency (width), deficit, and excess of conformal prediction intervals.
  arXiv  Detail & Related papers  (2023-02-23T18:57:14Z)
• Sample-Efficient Optimisation with Probabilistic Transformer Surrogates [66.98962321504085]
  This paper investigates the feasibility of employing state-of-the-art probabilistic transformers in Bayesian optimisation. We observe two drawbacks stemming from their training procedure and loss definition, hindering their direct deployment as proxies in black-box optimisation. We introduce two components: 1) a BO-tailored training prior supporting non-uniformly distributed points, and 2) a novel approximate posterior regulariser trading-off accuracy and input sensitivity to filter favourable stationary points for improved predictive performance.
  arXiv  Detail & Related papers  (2022-05-27T11:13:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.