Anytime-Valid Confirmation of Covariate Balance for Prespecified Corrections

• URL: http://arxiv.org/abs/2607.23157v1
• Date: Sat, 25 Jul 2026 11:21:17 GMT
• Title: Anytime-Valid Confirmation of Covariate Balance for Prespecified Corrections
• Abstract summary: We study anytime-valid confirmation of prespecified corrections from sequential unlabeled target inputs.<n>Experiments illustrate false-confirmation control, locality to the balancing-function class, KL-drift diagnostics, acceptable-region monitoring, finite-source effects.
• Score: 0.6599344783327052
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Many covariate-shift adaptation methods construct a correction $w(x)$, but users must still determine whether the corrected distribution is sufficiently balanced for the target stream. We study anytime-valid confirmation of prespecified corrections from sequential unlabeled target inputs. Our primary contribution is a procedure for confirming covariate balance. For a prespecified class of balancing functions and tolerances, time-uniform confidence sequences permit continuous monitoring and data-dependent stopping once all plausible target moments lie within their tolerance bands. If the correction is out of tolerance for at least one function, the probability of ever incorrectly confirming balance is at most the prescribed level. Upon stopping, the procedure yields a certificate local to the chosen functions and tolerances, yet providing an absolute downstream-adequacy statement that ordinary shift diagnostics generally do not. With finite source data, contracted bands preserve this guarantee while accounting for uncertainty in weighted source moments, whereas expanded bands support only compatibility diagnostics. As complementary information, we study a source-calibrated likelihood-ratio e-process whose KL-drift identity characterizes correction directions relative to the source. Under the source-reference distribution, the probability of ever crossing its evidence threshold is controlled, but crossing does not confirm balance. We also give an exponential-tilt test for departures beyond an acceptable correction region and deploy balance-confirmed corrections in weighted conformal prediction. Experiments illustrate false-confirmation control, locality to the balancing-function class, KL-drift diagnostics, acceptable-region monitoring, finite-source effects, and downstream conformal coverage under covariate shift.

Related papers

• Learning to Adapt and Calibrate: Score Distribution Alignment for Few-Shot Uncertainty Prediction in Medical VLMs [9.993859591441238]
  We propose a framework that reconciles supervised few-shot adaptation with conformal uncertainty estimation under non-exchangeability.<n> AlignCP learns a reweighted calibration distribution that reduces the score-level discrepancy between the labeled support set and the unlabeled query set.
  arXiv  Detail & Related papers  (2026-09-09T15:32:01Z)
• Beyond Marginal Validity: Finite-Sample Guarantees for Localized Conformal Prediction [25.59471163637912]
  Conformal prediction endows arbitrary black-box predictors with finite-sample, distribution-free marginal coverage.<n>We provide finite-sample guarantees for the realized localized set that jointly control conditional validity and oracle efficiency.<n>We analyze data-split learned scores: when the score targets a pivotal score, uniform local guarantees decompose into fixed-score calibration and uniform score-estimation errors.
  arXiv  Detail & Related papers  (2026-08-06T15:59:57Z)
• Denoised Conformal Alignment for Reliable Selection of Conditional Average Treatment Effect Predictions [10.945117945979566]
  We study reliable selection for black-box CATE predictors.<n>We construct doubly robust proxy errors from pseudo-outcomes.<n>We propose Denoised Conformal Alignment.
  arXiv  Detail & Related papers  (2026-07-03T10:00:28Z)
• Anytime-Valid Confirmation of Label-Shift Corrections [0.6599344783327052]
  In small-batch scientific deployments, labeled target outcomes may be too scarce for reliable shift estimation.<n>We show that the per-observation likelihood ratio between a label-shift-corrected predictive and the source predictive is a conditional e-value.
  arXiv  Detail & Related papers  (2026-06-12T02:03:21Z)
• Discovery of Hidden Miscalibration Regimes [52.452902154360565]
  Models may be systematically overconfident on some kinds of inputs and underconfident on others.<n>We define the corresponding miscalibration field and propose a diagnostic framework for estimating it.<n>Our approach learns a calibration-aware representation of the input space and estimates signed local miscalibration by kernel smoothing in the learned geometry.
  arXiv  Detail & Related papers  (2026-05-13T13:07:50Z)
• Robust Conditional Conformal Prediction via Branched Normalizing Flow [20.92399096303012]
  Conformal prediction constructs with marginal coverage guarantees under the assumption that the calibration and test distributions are identical.<n>We introduce Branched Normalizing Flow (BNF), a two-branch architecture that normalizes a test input to the calibration distribution and transforms the prediction set of the normalized input back to the test distribution.<n>BNF consistently improves conditional coverage robustness on nine datasets across a wide range of confidence levels.
  arXiv  Detail & Related papers  (2026-05-03T13:29:26Z)
• ConformaDecompose: Explaining Uncertainty via Calibration Localization [0.26385121748044166]
  Conformal Prediction provides distribution-free prediction intervals with guaranteed coverage.<n>Its reliance on a single global calibration threshold obscures the sources of uncertainty at the instance level.<n>We introduce an uncertainty-aware explainability framework that analyses the reducibility of calibration-induced uncertainty.
  arXiv  Detail & Related papers  (2026-04-29T19:58:15Z)
• LATA: Laplacian-Assisted Transductive Adaptation for Conformal Uncertainty in Medical VLMs [61.06744611795341]
  Medical vision-language models (VLMs) are strong zero-shot recognizers for medical imaging.<n>We propose texttttextbfLATA (Laplacian-Assisted Transductive Adaptation), a textittraining- and label-free refinement.<n>texttttextbfLATA sharpens zero-shot predictions without compromising exchangeability.
  arXiv  Detail & Related papers  (2026-02-19T16:45:38Z)
• Adaptive Individual Uncertainty under Out-Of-Distribution Shift with Expert-Routed Conformal Prediction [8.306260695214972]
  We introduce Trustworthy Expert Split-conformal with Scaled Estimation for Efficient Reliable Adaptive intervals (TESSERA)<n>TESSERA provides per-sample uncertainty with reliable coverage guarantee, informative and adaptive prediction interval widths that track the absolute error.<n>We evaluate on protein-ligand binding affinity prediction under both independent and identically distributed (i.i.d.) and scaffold-based out-of-distribution (OOD) splits.
  arXiv  Detail & Related papers  (2025-10-17T01:51:33Z)
• COIN: Uncertainty-Guarding Selective Question Answering for Foundation Models with Provable Risk Guarantees [51.5976496056012]
  COIN is an uncertainty-guarding selection framework that calibrates statistically valid thresholds to filter a single generated answer per question.<n>COIN estimates the empirical error rate on a calibration set and applies confidence interval methods to establish a high-probability upper bound on the true error rate.<n>We demonstrate COIN's robustness in risk control, strong test-time power in retaining admissible answers, and predictive efficiency under limited calibration data.
  arXiv  Detail & Related papers  (2025-06-25T07:04:49Z)
• Latent Veracity Inference for Identifying Errors in Stepwise Reasoning [78.29317733206643]
  We introduce Veracity Search (VS), a discrete search algorithm over veracity assignments.<n>It performs otherwise intractable inference in the posterior distribution over latent veracity values.<n>It generalizes VS, enabling accurate zero-shot veracity inference in novel contexts.
  arXiv  Detail & Related papers  (2025-05-17T04:16:36Z)
• Provably Reliable Conformal Prediction Sets in the Presence of Data Poisoning [53.42244686183879]
  Conformal prediction provides model-agnostic and distribution-free uncertainty quantification.<n>Yet, conformal prediction is not reliable under poisoning attacks where adversaries manipulate both training and calibration data.<n>We propose reliable prediction sets (RPS): the first efficient method for constructing conformal prediction sets with provable reliability guarantees under poisoning.
  arXiv  Detail & Related papers  (2024-10-13T15:37:11Z)
• Distribution-free uncertainty quantification for classification under label shift [105.27463615756733]
  We focus on uncertainty quantification (UQ) for classification problems via two avenues. We first argue that label shift hurts UQ, by showing degradation in coverage and calibration. We examine these techniques theoretically in a distribution-free framework and demonstrate their excellent practical performance.
  arXiv  Detail & Related papers  (2021-03-04T20:51:03Z)
• Distribution-free binary classification: prediction sets, confidence intervals and calibration [106.50279469344937]
  We study three notions of uncertainty quantification -- calibration, confidence intervals and prediction sets -- for binary classification in the distribution-free setting. We derive confidence intervals for binned probabilities for both fixed-width and uniform-mass binning. As a consequence of our 'tripod' theorems, these confidence intervals for binned probabilities lead to distribution-free calibration.
  arXiv  Detail & Related papers  (2020-06-18T14:17:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.