A prior-free blind detection of information leakage from model predictions

• URL: http://arxiv.org/abs/2606.11267v1
• Date: Tue, 09 Jun 2026 05:13:03 GMT
• Title: A prior-free blind detection of information leakage from model predictions
• Abstract summary: We give a decision-theoretic framework in which leakage diagnostics are functionals of the predicted-risk/outcome law.<n>A recalibrated leak matching an honest model's calibration and discrimination is indistinguishable from honest performance by emphany function of the predictions.<n>A near-deterministic subgroup produces a sustained unit-purity head that no legitimate predictor of a non-deterministic outcome can manufacture.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Data leakage -- contamination of a model with information unavailable at baseline -- is the dominant reproducibility failure in machine-learning-based science, yet detection tools require training code, external data, or domain expertise. None operates on the artifact an auditor most often holds: the model's output. We ask what can be decided about leakage from predictions and outcomes alone. We give a decision-theoretic framework in which leakage diagnostics are functionals of the predicted-risk/outcome law, parameterized by a threshold-weighting linked to proper scoring rules and decision-curve analysis. We prove a sharp impossibility: a recalibrated leak matching an honest model's calibration and discrimination is indistinguishable from honest performance by \emph{any} function of the predictions, so the broad class is detectable only against an externally supplied ceiling on achievable discrimination. We then prove what leakage cannot hide: a near-deterministic subgroup -- the signature of a near-label leak -- produces a sustained unit-purity head that no legitimate predictor of a non-deterministic outcome can manufacture, yielding a prior-free test. These results organize leakage into a trichotomy -- miscalibrated, broad-calibrated, and deterministic -- each with a matched detector and failure mode. We validate on UK Biobank using time-windowed comorbidity leakage with known, graded severity, measuring a detection floor of $Δ\cstar \approx 0.007$ on this endpoint, below which residual leakage is undetectable from output and too small to alter conclusions. The numerical floor is cohort- and endpoint-specific; the structural lesson is general: output-only detection fails where residual leakage is indistinguishable from an honestly stronger predictor. The test returns a verdict on a prediction vector in under a second on commodity hardware.

Related papers

• Physically Verifiable Evidence and LLM-Based Reporting for Bearing Fault Diagnosis [6.168311548428149]
  Diagnostic Evidence Network (DENet) is an encoder-agnostic multi-task framework extending the output to a structured evidence record.<n>It incurs no statistically significant accuracy cost, with a frequency error of about 6 Hz on 1,024-point segments.<n>It detects misclassifications with AUROC values of 0.970 and 0.871, and remains discriminative in the high-confidence regime.
  arXiv  Detail & Related papers  (2026-07-24T14:09:16Z)
• Conformal calibration and look-elsewhere effect in anomaly detection for new-physics searches [0.0]
  Machine-learned anomaly detection is reshaping searches for new physics, but it has outrun the statistics used to interpret it.<n>We propose a calibration layer, built on conformal prediction, that turns any anomaly score into a defensible significance.
  arXiv  Detail & Related papers  (2026-06-11T18:00:02Z)
• Testing the Test: Score-Direction Instability in Class-Split Anomaly Detection [56.362776482614976]
  Within-dataset class-split evaluation is widely used as a proxy for fully unconditional out-of-distribution anomaly detection.<n>We show that this protocol can become ill-posed when the held-out anomaly class overlaps the normal mixture in representation space.<n>We introduce a simple training-free diagnostic, neighborhood class leakage, and show that it predicts score-direction instability across Fashion-MNIST, CIFAR-10, and Imagenette.
  arXiv  Detail & Related papers  (2026-05-23T06:27:42Z)
• World Model Failure Classification and Anomaly Detection for Autonomous Inspection [23.48742973289626]
  We propose a hybrid framework that combines supervised failure classification with anomaly detection.<n>Our approach uses a world model backbone with compressed video inputs.<n>Experiments show over 90% accuracy in distinguishing between successes, failures, and OOD cases.
  arXiv  Detail & Related papers  (2026-02-18T04:41:14Z)
• Calibrated Prediction Set in Fault Detection with Risk Guarantees via Significance Tests [3.500936878570599]
  This paper proposes a novel fault detection method that integrates significance testing with the conformal prediction framework to provide formal risk guarantees.<n>The proposed method consistently achieves an empirical coverage rate at or above the nominal level ($1-alpha$)<n>The results reveal a controllable trade-off between the user-defined risk level ($alpha$) and efficiency, where higher risk tolerance leads to smaller average prediction set sizes.
  arXiv  Detail & Related papers  (2025-08-02T05:49:02Z)
• Fault detection and diagnosis for the engine electrical system of a space launcher based on a temporal convolutional autoencoder and calibrated classifiers [0.0]
  This paper outlines a first step toward developing an onboard fault detection and diagnostic capability for the next generation of reusable space launchers.<n>Unlike existing approaches in the literature, our solution is designed to meet a broader range of key requirements.<n>The proposed solution is based on a temporal convolutional autoencoder to automatically extract low-dimensional features from raw sensor data.
  arXiv  Detail & Related papers  (2025-07-17T11:50:29Z)
• COIN: Uncertainty-Guarding Selective Question Answering for Foundation Models with Provable Risk Guarantees [51.5976496056012]
  COIN is an uncertainty-guarding selection framework that calibrates statistically valid thresholds to filter a single generated answer per question.<n>COIN estimates the empirical error rate on a calibration set and applies confidence interval methods to establish a high-probability upper bound on the true error rate.<n>We demonstrate COIN's robustness in risk control, strong test-time power in retaining admissible answers, and predictive efficiency under limited calibration data.
  arXiv  Detail & Related papers  (2025-06-25T07:04:49Z)
• Conformal Segmentation in Industrial Surface Defect Detection with Statistical Guarantees [2.0257616108612373]
  In industrial settings, surface defects on steel can significantly compromise its service life and elevate potential safety risks.<n>Traditional defect detection methods predominantly rely on manual inspection, which suffers from low efficiency and high costs.<n>We develop a statistically rigorous threshold based on a user-defined risk level to identify high-probability defective pixels in test images.<n>We demonstrate robust and efficient control over the expected test set error rate across varying calibration-to-test ratios.
  arXiv  Detail & Related papers  (2025-04-24T16:33:56Z)
• Provably Reliable Conformal Prediction Sets in the Presence of Data Poisoning [53.42244686183879]
  Conformal prediction provides model-agnostic and distribution-free uncertainty quantification.<n>Yet, conformal prediction is not reliable under poisoning attacks where adversaries manipulate both training and calibration data.<n>We propose reliable prediction sets (RPS): the first efficient method for constructing conformal prediction sets with provable reliability guarantees under poisoning.
  arXiv  Detail & Related papers  (2024-10-13T15:37:11Z)
• Conservative Prediction via Data-Driven Confidence Minimization [70.93946578046003]
  In safety-critical applications of machine learning, it is often desirable for a model to be conservative. We propose the Data-Driven Confidence Minimization framework, which minimizes confidence on an uncertainty dataset.
  arXiv  Detail & Related papers  (2023-06-08T07:05:36Z)
• Unsupervised Anomaly Detection with Rejection [19.136286864839846]
  Anomaly detectors learn a decision boundary by employing intuitions, which are hard to verify in practice. A way to combat this is by allowing the detector to reject examples with high uncertainty. This requires employing a confidence metric that captures the distance to the decision boundary and setting a rejection threshold to reject low-confidence predictions.
  arXiv  Detail & Related papers  (2023-05-22T16:22:32Z)
• Dense Uncertainty Estimation via an Ensemble-based Conditional Latent Variable Model [68.34559610536614]
  We argue that the aleatoric uncertainty is an inherent attribute of the data and can only be correctly estimated with an unbiased oracle model. We propose a new sampling and selection strategy at train time to approximate the oracle model for aleatoric uncertainty estimation. Our results show that our solution achieves both accurate deterministic results and reliable uncertainty estimation.
  arXiv  Detail & Related papers  (2021-11-22T08:54:10Z)
• CertainNet: Sampling-free Uncertainty Estimation for Object Detection [65.28989536741658]
  Estimating the uncertainty of a neural network plays a fundamental role in safety-critical settings. In this work, we propose a novel sampling-free uncertainty estimation method for object detection. We call it CertainNet, and it is the first to provide separate uncertainties for each output signal: objectness, class, location and size.
  arXiv  Detail & Related papers  (2021-10-04T17:59:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.