Data Fusion with Distributional Equivalence Test-then-pool

• URL: http://arxiv.org/abs/2603.11867v2
• Date: Mon, 16 Mar 2026 08:30:10 GMT
• Title: Data Fusion with Distributional Equivalence Test-then-pool
• Abstract summary: Existing test-then-pool (TTP) procedures address this concern by testing for equality of control outcomes between historical and concurrent trials.<n>We develop a new TTP framework that fuses control arms while rigorously controlling the Type-I error rate of the final treatment effect test.
• Score: 5.5945020319921435
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Randomized controlled trials (RCTs) are the gold standard for causal inference, yet practical constraints often limit the size of the concurrent control arm. Borrowing control data from previous trials offers a potential efficiency gain, but naive borrowing can induce bias when historical and current populations differ. Existing test-then-pool (TTP) procedures address this concern by testing for equality of control outcomes between historical and concurrent trials before borrowing; however, standard implementations may suffer from reduced power or inadequate control of the Type-I error rate. We develop a new TTP framework that fuses control arms while rigorously controlling the Type-I error rate of the final treatment effect test. Our method employs kernel two-sample testing via maximum mean discrepancy (MMD) to capture distributional differences, and equivalence testing to avoid introducing uncontrolled bias, providing a more flexible and informative criterion for pooling. To ensure valid inference, we introduce partial bootstrap and partial permutation procedures for approximating null distributions in the presence of heterogeneous controls. We further establish the overall validity and consistency. We provide empirical studies demonstrating that the proposed approach achieves higher power than standard TTP methods while maintaining nominal error control, highlighting its value as a principled tool for leveraging historical controls in modern clinical trials.

Related papers

• Accelerating A/B-Tests with Counterfactual Estimation: Reducing Variance through Policy Overlap [8.291484471359633]
  We identify a critical inefficiency in the standard A/B-testing protocol.<n>When a treatment and control policy agree on an action, the resulting outcome contributes noise but no signal regarding the treatment effect.<n>We propose a novel experimental protocol that exploits this policy overlap to accelerate experimentation.
  arXiv  Detail & Related papers  (2026-07-16T06:08:52Z)
• Everywhere Valid Bounds on False Discovery Proportions in Conformal Inference [19.796377044066286]
  Conformal inference is often used to select test samples whose conformal p-values fall below a threshold.<n>The quality of such methods is often measured by the false discovery proportion (FDP), defined as the fraction of incorrect selections.<n>This paper establishes finite-sample, distribution-free upper bounds on the FDP that hold simultaneously over all possible rejection thresholds.
  arXiv  Detail & Related papers  (2026-05-20T05:24:39Z)
• Towards Anytime-Valid Statistical Watermarking [63.02116925616554]
  We develop the first e-value-based watermarking framework, Anchored E-Watermarking, that unifies optimal sampling with anytime-valid inference.<n>Our framework can significantly enhance sample efficiency, reducing the average token budget required for detection by 13-15% relative to state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-02-19T18:32:26Z)
• Testing For Distribution Shifts with Conditional Conformal Test Martingales [20.369245689839822]
  We propose a sequential test for detecting arbitrary distribution shifts that allows conformal test martingales (CTMs) to work under a fixed, reference-conditional setting.<n>Existing CTM detectors construct test martingales by continually growing a reference set with each incoming sample, using it to assess how atypical the new sample is relative to past observations.<n>While this design yields anytime-valid type-I error control, it suffers from test-time contamination: after a change, post-shift observations enter the reference set and dilute the evidence for distribution shift, increasing detection delay and reducing power.<n>Our method detects shifts faster than standard
  arXiv  Detail & Related papers  (2026-02-14T18:47:26Z)
• Reliable Selection of Heterogeneous Treatment Effect Estimators [10.969092501407635]
  We study the problem of selecting the best heterogeneous treatment effect (HTE) estimator from a collection of candidates in settings where the treatment effect is fundamentally unobserved.<n>We cast estimator selection as a multiple testing problem and introduce a ground-truth-free procedure based on a cross-fitted, exponentially weighted test statistic.
  arXiv  Detail & Related papers  (2025-11-23T14:15:50Z)
• COIN: Uncertainty-Guarding Selective Question Answering for Foundation Models with Provable Risk Guarantees [51.5976496056012]
  COIN is an uncertainty-guarding selection framework that calibrates statistically valid thresholds to filter a single generated answer per question.<n>COIN estimates the empirical error rate on a calibration set and applies confidence interval methods to establish a high-probability upper bound on the true error rate.<n>We demonstrate COIN's robustness in risk control, strong test-time power in retaining admissible answers, and predictive efficiency under limited calibration data.
  arXiv  Detail & Related papers  (2025-06-25T07:04:49Z)
• Max-Rank: Efficient Multiple Testing for Conformal Prediction [43.56898111853698]
  Multiple hypothesis testing (MHT) frequently arises in scientific inquiries, and concurrent testing of multiple hypotheses inflates the risk of Type-I errors or false positives.<n>This paper addresses MHT in the context of conformal prediction, a flexible framework for predictive uncertainty quantification.<n>We introduce $textttmax-rank$, a novel correction that exploits dependencies whilst efficiently controlling the family-wise error rate.
  arXiv  Detail & Related papers  (2023-11-17T22:44:22Z)
• On Pitfalls of Test-Time Adaptation [82.8392232222119]
  Test-Time Adaptation (TTA) has emerged as a promising approach for tackling the robustness challenge under distribution shifts. We present TTAB, a test-time adaptation benchmark that encompasses ten state-of-the-art algorithms, a diverse array of distribution shifts, and two evaluation protocols.
  arXiv  Detail & Related papers  (2023-06-06T09:35:29Z)
• Sequential Predictive Two-Sample and Independence Testing [114.4130718687858]
  We study the problems of sequential nonparametric two-sample and independence testing. We build upon the principle of (nonparametric) testing by betting.
  arXiv  Detail & Related papers  (2023-04-29T01:30:33Z)
• Sequential Kernelized Independence Testing [77.237958592189]
  We design sequential kernelized independence tests inspired by kernelized dependence measures.<n>We demonstrate the power of our approaches on both simulated and real data.
  arXiv  Detail & Related papers  (2022-12-14T18:08:42Z)
• Error-based Knockoffs Inference for Controlled Feature Selection [49.99321384855201]
  We propose an error-based knockoff inference method by integrating the knockoff features, the error-based feature importance statistics, and the stepdown procedure together. The proposed inference procedure does not require specifying a regression model and can handle feature selection with theoretical guarantees.
  arXiv  Detail & Related papers  (2022-03-09T01:55:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.