Cross-Domain Uncertainty Quantification for Selective Prediction: A Comprehensive Bound Ablation with Transfer-Informed Betting

• URL: http://arxiv.org/abs/2603.08907v1
• Date: Mon, 09 Mar 2026 20:25:18 GMT
• Title: Cross-Domain Uncertainty Quantification for Selective Prediction: A Comprehensive Bound Ablation with Transfer-Informed Betting
• Abstract summary: We present a comprehensive ablation of nine finite-sample bound families for selective prediction with risk control.<n>Our main theoretical contribution is Transfer-Informed Betting (TIB), which warm-starts the WSR wealth process using a source domain's risk profile.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present a comprehensive ablation of nine finite-sample bound families for selective prediction with risk control, combining concentration inequalities (Hoeffding, Empirical Bernstein, Clopper-Pearson, Wasserstein DRO, CVaR) with multiple-testing corrections (union bound, Learn Then Test fixed-sequence) and betting-based confidence sequences (WSR). Our main theoretical contribution is Transfer-Informed Betting (TIB), which warm-starts the WSR wealth process using a source domain's risk profile, achieving tighter bounds in data-scarce settings with a formal dominance guarantee. We prove that the TIB wealth process remains a valid supermartingale under all source-target divergences, that TIB dominates standard WSR when domains match, and that no data-independent warm-start can achieve better convergence. The combination of betting-based confidence sequences, LTT monotone testing, and cross-domain transfer is, to our knowledge, a three-way novelty not present in the literature. We evaluate all nine bound families on four benchmarks-MASSIVE (n=1,102), NyayaBench (n=280), CLINC-150 (n=22.5K), and Banking77 (n=13K)-across 18 (alpha, delta) configurations. On MASSIVE at alpha=0.10, LTT eliminates the ln(K) union-bound penalty, achieving 94.0% guaranteed coverage versus 73.8% for Hoeffding-a 27% relative improvement. On NyayaBench, where the small calibration set makes Hoeffding-family bounds infeasible below alpha=0.20, Transfer-Informed Betting achieves 18.5% coverage at alpha=0.10, a 5.4x improvement over LTT + Hoeffding. We additionally compare with split-conformal prediction, showing that conformal methods produce prediction sets (avg. 1.67 classes) whereas selective prediction provides single-prediction risk guarantees. We apply these methods to agentic caching systems, formalizing a progressive trust model where the guarantee determines when cached responses can be served autonomously.

Related papers

• Post-Hoc Trajectory-Risk Certification for Modular LLM-Based Security Agents [1.4915002678801434]
  We show that Bonferroni allocation is distribution-free but conservative under correlated errors.<n> coarse-to-fine label selection can create near-perfect measured correlation without learned dependence.<n>A modular certificate using per-stage certificates and a pairwise overlap bound yields a positive average gain of 0.6%.
  arXiv  Detail & Related papers  (2026-08-04T23:48:36Z)
• Cross-Fitted Residual Utility for Primary-Preserving Cognitive Decision Correction in Automatic Modulation Classification [1.547549252134621]
  We study a post-inference problem through cross-fitted residual utility and a primary-preserving cognitive decision policy.<n>A structured KAN-Fourier classifier supplies the default probability, while neural and non-neural candidates provide observable evidence.<n>On RMLA, RMLB, and HISAR, the complete system improves overall accuracy from 63.632% to 66.332%, 65.161% to 66.168%, and 77.769% to 79.867%.
  arXiv  Detail & Related papers  (2026-08-03T11:07:37Z)
• When Can Conformal Risk Control Certify LLM Outputs? Bounds, Impossibility, and Adaptation for Structured Generation [0.11280931253550518]
  Large language models (LLMs) deployed for structured generation (NER, extraction, QA, and classification) lack formal reliability guarantees.<n>We characterize when conformal risk control (CRC) can certify structured LLM outputs and when it provably cannot.
  arXiv  Detail & Related papers  (2026-06-27T19:25:07Z)
• Architecture-Adaptive Uncertainty Fusion for Deepfake Detection [0.9940728137241215]
  Deepfake detection systems achieve near-perfect accuracy on benchmarks, yet forensic deployment demands reliable prediction uncertainty.<n>We propose Correlation-d Fusion (COF), an architecture-adaptive framework that fuses five complementary uncertainty sources.<n>COF requires no model modifications and only 42 s of weight optimization, compared to 20--45 h for a 5-model Deep Ensemble.
  arXiv  Detail & Related papers  (2026-06-04T19:28:55Z)
• Unveiling the Entropy Dynamics of Chain-of-Thought Reasoning [81.57028614960576]
  We show that the Confidence Region possesses two critical properties: 1) High Reliability -- answers in the confidence region become highly accurate and stable, and 2) High Redundancy -- models generate unnecessary tokens long after reaching the correct answer.<n>These properties unlock more efficient and reliable inference strategies.
  arXiv  Detail & Related papers  (2026-06-01T10:11:14Z)
• Byzantine-Robust and Differentially Private Federated Optimization under Weaker Assumptions [89.52532304099522]
  Federated Learning (FL) enables clients to collaboratively train a shared model without centralizing their raw data, offering an inherent level of privacy.<n> gradients and model updates can still leak sensitive information, while malicious servers may mount adversarial attacks such as Byzantine manipulation.<n>These vulnerabilities highlight the need to address differential privacy (DP) and Byzantine robustness within a unified framework.<n>We propose Byz-Clip21-SGD2M, a new algorithm that integrates robust aggregation with double momentum and carefully designed clipping.
  arXiv  Detail & Related papers  (2026-03-24T17:39:09Z)
• Bayesian Transformer for Probabilistic Load Forecasting in Smart Grids [0.0]
  This study proposes a Bayesian Transformer framework that integrates three complementary uncertainty mechanisms into a PatchTST backbone.<n>A seven-level multi-quantile pinball-loss prediction head and post-training isotonic regression calibration produce sharp, near-nominally covered prediction intervals.<n>On the primary benchmark (PJM, H=24h), BT achieves a CRPS of 0.0289, improving 7.4% over Deep Ensembles and 29.9% over the deterministic LSTM.
  arXiv  Detail & Related papers  (2026-03-09T02:39:51Z)
• ST-BCP: Tightening Coverage Bound for Backward Conformal Prediction via Non-Conformity Score Transformation [18.272247805086284]
  Conformal Prediction (CP) provides a statistical framework for uncertainty quantification that constructs prediction sets with coverage guarantees.<n>BCP inverts this paradigm by enforcing a predefined upper bound on set size and estimating the resulting coverage guarantee.<n>We introduce ST-BCP, a novel method that introduces a data-dependent transformation of nonconformity scores to narrow the coverage gap.
  arXiv  Detail & Related papers  (2026-02-02T07:18:35Z)
• Semantic-Constrained Federated Aggregation: Convergence Theory and Privacy-Utility Bounds for Knowledge-Enhanced Distributed Learning [0.0]
  We introduce Semantic-Constrained Federated Aggregation (SCFA), a theoretically-grounded framework incorporating domain knowledge constraints into distributed optimization.<n>We prove SCFA convergence rate O(1/sqrt(T) + rho) where rho represents constraint violation rate, establishing the first convergence theory for constraint-based federated learning.<n>We validate our framework on manufacturing predictive maintenance using Bosch production data with 1.18 million samples and 968 sensor features.
  arXiv  Detail & Related papers  (2025-12-12T04:29:29Z)
• LEC: Linear Expectation Constraints for False-Discovery Control in Selective Prediction and Routing Systems [95.35293543918762]
  Large language models (LLMs) often generate unreliable answers, while uncertainty methods fail to fully distinguish correct from incorrect predictions.<n>We address this issue through the lens of false discovery rate (FDR) control, ensuring that among all accepted predictions, the proportion of errors does not exceed a target risk level.<n>We propose LEC, which reinterprets selective prediction as a constrained decision problem by enforcing a Linear Expectation Constraint.
  arXiv  Detail & Related papers  (2025-12-01T11:27:09Z)
• Formal Models and Convergence Analysis for Context-Aware Security Verification [0.0]
  We present a formal framework for context-aware security verification that establishes provable guarantees for ML-enhanced adaptive systems.<n>We introduce context-completeness - a new security property - and prove: (1) sample complexity bounds showing when adaptive verification succeeds, (2) information-theoretic limits relating context richness to detection capability, and (3) convergence guarantees for ML-based payload generators.
  arXiv  Detail & Related papers  (2025-10-14T12:21:36Z)
• Conformal Prediction for Privacy-Preserving Machine Learning [83.88591755871734]
  Using AES-encrypted variants of the MNIST dataset, we demonstrate that Conformal Prediction methods remain effective even when applied directly in the encrypted domain.<n>Our work sets a foundation for principled uncertainty quantification in secure, privacy-aware learning systems.
  arXiv  Detail & Related papers  (2025-07-13T15:29:14Z)
• COIN: Uncertainty-Guarding Selective Question Answering for Foundation Models with Provable Risk Guarantees [51.5976496056012]
  COIN is an uncertainty-guarding selection framework that calibrates statistically valid thresholds to filter a single generated answer per question.<n>COIN estimates the empirical error rate on a calibration set and applies confidence interval methods to establish a high-probability upper bound on the true error rate.<n>We demonstrate COIN's robustness in risk control, strong test-time power in retaining admissible answers, and predictive efficiency under limited calibration data.
  arXiv  Detail & Related papers  (2025-06-25T07:04:49Z)
• Robust Conformal Prediction with a Single Binary Certificate [58.450154976190795]
  Conformal prediction (CP) converts any model's output to prediction sets with a guarantee to cover the true label with (adjustable) high probability.<n>We propose a robust conformal prediction that produces smaller sets even with significantly lower MC samples.
  arXiv  Detail & Related papers  (2025-03-07T08:41:53Z)
• Equal Opportunity of Coverage in Fair Regression [50.76908018786335]
  We study fair machine learning (ML) under predictive uncertainty to enable reliable and trustworthy decision-making. We propose Equal Opportunity of Coverage (EOC) that aims to achieve two properties: (1) coverage rates for different groups with similar outcomes are close, and (2) the coverage rate for the entire population remains at a predetermined level.
  arXiv  Detail & Related papers  (2023-11-03T21:19:59Z)
• Learning Calibrated Uncertainties for Domain Shift: A Distributionally Robust Learning Approach [150.8920602230832]
  We propose a framework for learning calibrated uncertainties under domain shifts. In particular, the density ratio estimation reflects the closeness of a target (test) sample to the source (training) distribution. We show that our proposed method generates calibrated uncertainties that benefit downstream tasks.
  arXiv  Detail & Related papers  (2020-10-08T02:10:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.