Empirically Calibrated Conditional Independence Tests

• URL: http://arxiv.org/abs/2602.21036v1
• Date: Tue, 24 Feb 2026 15:56:19 GMT
• Title: Empirically Calibrated Conditional Independence Tests
• Abstract summary: Conditional independence tests (CIT) are widely used for causal discovery and feature selection.<n>We propose Empirically Calibrated Conditional Independence Tests (ECCIT) to measure and correct for miscalibration.
• Score: 2.28126966226433
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Conditional independence tests (CIT) are widely used for causal discovery and feature selection. Even with false discovery rate (FDR) control procedures, they often fail to provide frequentist guarantees in practice. We highlight two common failure modes: (i) in small samples, asymptotic guarantees for many CITs can be inaccurate and even correctly specified models fail to estimate the noise levels and control the error, and (ii) when sample sizes are large but models are misspecified, unaccounted dependencies skew the test's behavior and fail to return uniform p-values under the null. We propose Empirically Calibrated Conditional Independence Tests (ECCIT), a method that measures and corrects for miscalibration. For a chosen base CIT (e.g., GCM, HRT), ECCIT optimizes an adversary that selects features and response functions to maximize a miscalibration metric. ECCIT then fits a monotone calibration map that adjusts the base-test p-values in proportion to the observed miscalibration. Across empirical benchmarks on synthetic and real data, ECCIT achieves valid FDR with higher power than existing calibration strategies while remaining test agnostic.

Related papers

• Robust Conformalized Selection with Noisy Responses [23.306273801765297]
  Conformalized selection has been widely applied to select high-quality candidates from large datasets.<n>We formulate the above tasks as selecting candidates with true predicted labels or with responses exceeding certain values.<n>We propose Robust Conformalized Selection (RCS), a unified framework for selective classification with valid FDR control under general label contamination.
  arXiv  Detail & Related papers  (2026-07-25T01:51:46Z)
• Fast Nonparametric Conditional Independence Testing via Two-Stage Regression [1.8275108630751837]
  We introduce BLITZ (Broad-to-Local Independence Testing via residualiZation), a nonparametric conditional independence test designed to run well under a second.<n>BLITZ removes broad smooth dependence on the conditioning set using low-order regression, then applies a small nonlinear feature map and residualizes those features with shallow tree regressions.<n>We show theoretically that the two-stage design reduces the effective complexity faced by the tree residualizers, allowing shallow trees to control residual conditional-mean bias while avoiding excessive overfitting.
  arXiv  Detail & Related papers  (2026-06-16T14:55:00Z)
• Provable Joint Decontamination for Benchmarking Multiple Large Language Models [19.929035827959822]
  We formalize benchmark decontamination as a joint selection problem and propose Joint Envelope Conformal Selection (JECS)<n>JECS computes per-model conformal p-values, aggregates them by the per-item maximum, and reconstructs a conservative envelope of the max-p null distribution.<n>Experiments across various models and benchmarks demonstrate that JECS achieves higher power than the max-p baseline while consistently maintaining the target GCR control.
  arXiv  Detail & Related papers  (2026-05-20T09:16:39Z)
• Multimodal Learning on Low-Quality Data with Conformal Predictive Self-Calibration [72.0672328514289]
  Multimodal learning often grapples with the challenge of low-quality data.<n>We propose a unified framework, termed Conformal Predictive Self-Calibration.<n>Our framework consistently outperforms existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-05-05T14:48:52Z)
• The Generalised Kernel Covariance Measure [12.90382979353427]
  We consider the problem of conditional independence (CI) testing and adopt a kernel-based approach.<n> Kernel-based CI tests embed variables in reproducing kernel Hilbert spaces, regress their embeddings on the conditioning variables, and test the resulting residuals for marginal independence.<n>Existing methods rely heavily on kernel ridge regression, which is computationally expensive when properly tuned and yields poorly calibrated tests when left untuned.<n>We propose the Generalised Kernel Covariance Measure (GKCM), a regression-model-agnostic kernel-based CI test framework.
  arXiv  Detail & Related papers  (2026-04-04T12:55:00Z)
• Adversarial Stress Tests for Quantum Certification [0.0]
  Apparent violations of classical benchmarks need not signal genuinely non-classical behaviour.<n>We formalise a protocol-agnostic alignment principle based on a martingale-safe lower confidence bound.<n>We show that adaptive learning-based classical agents do not enlarge the admissible classical set.
  arXiv  Detail & Related papers  (2026-03-13T03:49:58Z)
• Calibratable Disambiguation Loss for Multi-Instance Partial-Label Learning [53.9713678229744]
  Multi-instance partial-label learning (MIPL) is a weakly supervised framework that addresses the challenges of inexact supervision in both instance and label spaces.<n>Existing MIPL approaches often suffer from poor calibration, undermining reliability.<n>We propose a plug-and-play calibratable disambiguation loss (CDL) that simultaneously improves classification accuracy and calibration performance.
  arXiv  Detail & Related papers  (2025-12-19T16:58:31Z)
• LEC: Linear Expectation Constraints for False-Discovery Control in Selective Prediction and Routing Systems [95.35293543918762]
  Large language models (LLMs) often generate unreliable answers, while uncertainty methods fail to fully distinguish correct from incorrect predictions.<n>We address this issue through the lens of false discovery rate (FDR) control, ensuring that among all accepted predictions, the proportion of errors does not exceed a target risk level.<n>We propose LEC, which reinterprets selective prediction as a constrained decision problem by enforcing a Linear Expectation Constraint.
  arXiv  Detail & Related papers  (2025-12-01T11:27:09Z)
• Fast and Scalable Score-Based Kernel Calibration Tests [28.41894759475933]
  We introduce the Kernel Conditional Stein Discrepancy test (KCCSD test), a non-parametric, kernel-based test for assessing the calibration of probabilistic models with well-defined scores.<n>In contrast to previous methods, our test avoids the need for possibly expensive expectation approximations while providing control over its type-I error.
  arXiv  Detail & Related papers  (2025-10-16T14:11:14Z)
• COIN: Uncertainty-Guarding Selective Question Answering for Foundation Models with Provable Risk Guarantees [51.5976496056012]
  COIN is an uncertainty-guarding selection framework that calibrates statistically valid thresholds to filter a single generated answer per question.<n>COIN estimates the empirical error rate on a calibration set and applies confidence interval methods to establish a high-probability upper bound on the true error rate.<n>We demonstrate COIN's robustness in risk control, strong test-time power in retaining admissible answers, and predictive efficiency under limited calibration data.
  arXiv  Detail & Related papers  (2025-06-25T07:04:49Z)
• A Sample Efficient Conditional Independence Test in the Presence of Discretization [54.047334792855345]
  Conditional Independence (CI) tests directly to discretized data can lead to incorrect conclusions.<n>Recent advancements have sought to infer the correct CI relationship between the latent variables through binarizing observed data.<n>Motivated by this, this paper introduces a sample-efficient CI test that does not rely on the binarization process.
  arXiv  Detail & Related papers  (2025-06-10T12:41:26Z)
• Score-based Generative Modeling for Conditional Independence Testing [35.0533359302886]
  We propose a novel CI testing method via score-based generative modeling, which achieves precise Type I error control and strong testing power.<n>We theoretically establish the error bound of conditional distributions modeled by score-based generative models and prove the validity of our CI tests.
  arXiv  Detail & Related papers  (2025-05-29T10:10:46Z)
• Error-quantified Conformal Inference for Time Series [55.11926160774831]
  Uncertainty quantification in time series prediction is challenging due to the temporal dependence and distribution shift on sequential data.<n>We propose itError-quantified Conformal Inference (ECI) by smoothing the quantile loss function.<n>ECI can achieve valid miscoverage control and output tighter prediction sets than other baselines.
  arXiv  Detail & Related papers  (2025-02-02T15:02:36Z)
• Testing Conditional Mean Independence Using Generative Neural Networks [8.323172773256449]
  We introduce a novel population CMI measure and a bootstrap model-based testing procedure.<n>Deep generative neural networks are used to estimate the conditional mean functions involved in the population measure.
  arXiv  Detail & Related papers  (2025-01-28T23:35:24Z)
• Conditional Testing based on Localized Conformal p-values [5.6779147365057305]
  We define the localized conformal p-values by inverting prediction intervals and prove their theoretical properties. These defined p-values are then applied to several conditional testing problems to illustrate their practicality.
  arXiv  Detail & Related papers  (2024-09-25T11:30:14Z)
• Uncertainty-Calibrated Test-Time Model Adaptation without Forgetting [65.21599711087538]
  Test-time adaptation (TTA) seeks to tackle potential distribution shifts between training and test data by adapting a given model w.r.t. any test sample.<n>Prior methods perform backpropagation for each test sample, resulting in unbearable optimization costs to many applications.<n>We propose an Efficient Anti-Forgetting Test-Time Adaptation (EATA) method which develops an active sample selection criterion to identify reliable and non-redundant samples.
  arXiv  Detail & Related papers  (2024-03-18T05:49:45Z)
• Model-based causal feature selection for general response types [8.228587135343071]
  Invariant causal prediction (ICP) is a method for causal feature selection which requires data from heterogeneous settings. We develop transformation-model (TRAM) based ICP, allowing for continuous, categorical, count-type, and uninformatively censored responses. We provide an open-source R package 'tramicp' and evaluate our approach on simulated data and in a case study investigating causal features of survival in critically ill patients.
  arXiv  Detail & Related papers  (2023-09-22T12:42:48Z)
• Sequential Kernelized Independence Testing [77.237958592189]
  We design sequential kernelized independence tests inspired by kernelized dependence measures.<n>We demonstrate the power of our approaches on both simulated and real data.
  arXiv  Detail & Related papers  (2022-12-14T18:08:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.