Significance-First Splitting: Aligning Treatment Heterogeneity Detection with Honest Estimation

• URL: http://arxiv.org/abs/2607.03999v1
• Date: Sat, 04 Jul 2026 19:50:35 GMT
• Title: Significance-First Splitting: Aligning Treatment Heterogeneity Detection with Honest Estimation
• Abstract summary: Estimating heterogeneous treatment effects (CATE) requires simultaneously detecting effect modification and quantifying estimation uncertainty.<n>Existing tree-based methods make an uneasy trade-off: significance-based approaches identify subgroup interactions directly but lack valid inference.<n>We introduce a hybrid algorithm that fuses significance-based splitting with honest sample-splitting and cross-validation.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Estimating heterogeneous treatment effects (CATE) requires simultaneously detecting effect modification and quantifying estimation uncertainty. Existing tree-based methods make an uneasy trade-off: significance-based approaches (Radcliffe and Surry 2011) identify subgroup interactions directly but lack valid inference; honest causal trees (Athey and Imbens 2016) deliver nominal confidence interval coverage but use outcome-agnostic splitting criteria that sacrifice interaction sensitivity. We introduce a hybrid algorithm that fuses significance-based splitting with honest sample-splitting and cross-validation. Our splitting criterion uses the squared $t$-statistic for the treatment $\times$ side interaction ($t^2$), which is shown to be directly aligned with the honest $\text{EMSE}_τ$ criterion when the interaction is strong. Post-hoc honest cross-validation selects the cost-complexity penalty, giving a single principled estimator with nominal CI coverage at the leaf level. For forests, we retain bootstrap count vectors to enable an infinitesimal jackknife (IJ) variance estimate of Monte-Carlo convergence rather than formal pointwise inference. On the three synthetic designs from (Athey and Imbens 2016) the single tree achieves approximately 90\% leaf-average CI coverage at the 90\% nominal level across all three designs (200 replications each); on the Criteo and Starbucks uplift datasets we match Qini coefficient performance of S- and T-learner baselines. An open-source Python package with reproducible seeds, sklearn-compatible API, and full test coverage accompanies this work (https://codeberg.org/hadjipantelis/rattus).

Related papers

• Semiparametric Inference for Conditional Shapley Feature Importance [0.0]
  This paper studies the conditional formulation, in which out-of-coalition features are integrated out under their true conditional distribution.<n>We propose a one-step estimator with K-fold cross-fitting and a U-statistic correction of the squared loss interval.<n>A Pinsker-type bound quantifies the bias from misspecifying the working copula class, while vine copulas keep conditional sampling tractable.
  arXiv  Detail & Related papers  (2026-09-09T15:19:24Z)
• Learning Auditable Classifier Models: Source-Disjoint Tree Ensembles [0.0]
  Predictive models in clinical and regulated settings must be accurate and fully auditable.<n>We introduce Residual Pattern Tree Ensemble (RPTE), a three-stage learning approach.<n>RPTE builds on three key principles: bounded feature budget, source disjointness, and separate coefficient estimation.
  arXiv  Detail & Related papers  (2026-08-16T12:56:49Z)
• How Many Trees in a Random Forest? A Revisited Approach with Plateau Search and Optuna Integration [0.30586855806896046]
  We propose an integrated triplet-based plateau-search algorithm for HPO for Random Forest.<n>The method adaptively tracks a near-minimal sufficient ensemble size by monitoring relative changes in the out-of-bag score.<n>Experiments show that the selected number of trees can differ substantially from the common absolute.
  arXiv  Detail & Related papers  (2026-06-02T12:10:43Z)
• TabSHAP [5.3259362673757735]
  Large Language Models are emerging as powerful alternatives to traditional tree-based models.<n>Existing methods often rely on global linear proxies or scalar probability shifts that fail to capture the model's full probabilistic uncertainty.<n>We introduce TabSHAP, a model-agnostic interpretability framework designed to directly attribute local query decision logic.
  arXiv  Detail & Related papers  (2026-04-22T22:04:45Z)
• Almost Asymptotically Optimal Active Clustering Through Pairwise Observations [59.20614082241528]
  We propose a new analysis framework for clustering $M$ items into an unknown number of $K$ distinct groups using noisy and actively collected responses.<n>We establish a fundamental lower bound on the expected number of queries needed to achieve a desired confidence in the accuracy of the clustering.<n>We develop a computationally feasible variant of the Generalized Likelihood Ratio statistic and show that its performance gap to the lower bound can be accurately empirically estimated.
  arXiv  Detail & Related papers  (2026-02-05T14:16:47Z)
• Ensemble Threshold Calibration for Stable Sensitivity Control [0.0]
  We present an end-to-end framework that achieves exact recall with sub-percent variance over tens of millions of geometry pairs.<n>Our approach consistently hits a recall target within a small error, decreases redundant verifications relative to other calibrations, and runs end-to-end on a single TPU v3 core.
  arXiv  Detail & Related papers  (2025-10-02T15:22:28Z)
• Cross-World Assumption and Refining Prediction Intervals for Individual Treatment Effects [6.083038976289835]
  For high-stakes decision-making, individual treatment effect estimates must be accompanied by valid prediction intervals.<n>For high-stakes decision-making, individual treatment effect estimates must be accompanied by valid prediction intervals.
  arXiv  Detail & Related papers  (2025-07-16T18:58:18Z)
• Semiparametric conformal prediction [79.6147286161434]
  We construct a conformal prediction set accounting for the joint correlation structure of the vector-valued non-conformity scores.<n>We flexibly estimate the joint cumulative distribution function (CDF) of the scores.<n>Our method yields desired coverage and competitive efficiency on a range of real-world regression problems.
  arXiv  Detail & Related papers  (2024-11-04T14:29:02Z)
• Improving Bias Correction Standards by Quantifying its Effects on Treatment Outcomes [54.18828236350544]
  Propensity score matching (PSM) addresses selection biases by selecting comparable populations for analysis. Different matching methods can produce significantly different Average Treatment Effects (ATE) for the same task, even when meeting all validation criteria. To address this issue, we introduce a novel metric, A2A, to reduce the number of valid matches.
  arXiv  Detail & Related papers  (2024-07-20T12:42:24Z)
• Mitigating LLM Hallucinations via Conformal Abstention [70.83870602967625]
  We develop a principled procedure for determining when a large language model should abstain from responding in a general domain. We leverage conformal prediction techniques to develop an abstention procedure that benefits from rigorous theoretical guarantees on the hallucination rate (error rate) Experimentally, our resulting conformal abstention method reliably bounds the hallucination rate on various closed-book, open-domain generative question answering datasets.
  arXiv  Detail & Related papers  (2024-04-04T11:32:03Z)
• The Lipschitz-Variance-Margin Tradeoff for Enhanced Randomized Smoothing [85.85160896547698]
  Real-life applications of deep neural networks are hindered by their unsteady predictions when faced with noisy inputs and adversarial attacks. We show how to design an efficient classifier with a certified radius by relying on noise injection into the inputs. Our novel certification procedure allows us to use pre-trained models with randomized smoothing, effectively improving the current certification radius in a zero-shot manner.
  arXiv  Detail & Related papers  (2023-09-28T22:41:47Z)
• Will My Robot Achieve My Goals? Predicting the Probability that an MDP Policy Reaches a User-Specified Behavior Target [56.99669411766284]
  As an autonomous system performs a task, it should maintain a calibrated estimate of the probability that it will achieve the user's goal. This paper considers settings where the user's goal is specified as a target interval for a real-valued performance summary. We compute the probability estimates by inverting conformal prediction.
  arXiv  Detail & Related papers  (2022-11-29T18:41:20Z)
• Tree-Values: selective inference for regression trees [0.0]
  A naive approach to inference that does not account for the fact that the tree was estimated from the data will not achieve standard guarantees. We propose a selective inference framework for conducting inference on a fitted CART tree.
  arXiv  Detail & Related papers  (2021-06-15T00:25:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.