Measuring Variable Importance in Heterogeneous Treatment Effects with Confidence

• URL: http://arxiv.org/abs/2408.13002v2
• Date: Fri, 07 Feb 2025 17:35:23 GMT
• Title: Measuring Variable Importance in Heterogeneous Treatment Effects with Confidence
• Authors: Joseph Paillard, Angel Reyero Lobo, Vitaliy Kolodyazhniy, Bertrand Thirion, Denis A. Engemann,
• Abstract summary: Causal machine learning holds promise for estimating individual treatment effects from complex data.<n>We propose PermuCATE, an algorithm based on the Conditional Permutation Importance (CPI) method.<n>We empirically demonstrate the benefits of PermuCATE in simulated and real-world health datasets.
• Score: 33.12963161545068
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Causal machine learning (ML) holds promise for estimating individual treatment effects from complex data. For successful real-world applications using machine learning methods, it is of paramount importance to obtain reliable insights into which variables drive heterogeneity in the response to treatment. We propose PermuCATE, an algorithm based on the Conditional Permutation Importance (CPI) method, for statistically rigorous global variable importance assessment in the estimation of the Conditional Average Treatment Effect (CATE). Theoretical analysis of the finite sample regime and empirical studies show that PermuCATE has lower variance than the Leave-One-Covariate-Out (LOCO) reference method and provides a reliable measure of variable importance. This property increases statistical power, which is crucial for causal inference in the limited-data regime common to biomedical applications. We empirically demonstrate the benefits of PermuCATE in simulated and real-world health datasets, including settings with up to hundreds of correlated variables.

Related papers

• Statistical Learning for Heterogeneous Treatment Effects: Pretraining, Prognosis, and Prediction [40.96453902709292]
  We propose pretraining strategies that leverage a phenomenon in real-world applications. In medicine, components of the same biological signaling pathways frequently influence both baseline risk and treatment response. We use this structure to incorporate "side information" and develop models that can exploit synergies between risk prediction and causal effect estimation.
  arXiv  Detail & Related papers  (2025-05-01T05:12:14Z)
• Estimating Individual Dose-Response Curves under Unobserved Confounders from Observational Data [6.166869525631879]
  We present ContiVAE, a novel framework for estimating causal effects of continuous treatments, measured by individual dose-response curves. We show that ContiVAE outperforms existing methods by up to 62%, demonstrating its robustness and flexibility.
  arXiv  Detail & Related papers  (2024-10-21T07:24:26Z)
• A Gradient Analysis Framework for Rewarding Good and Penalizing Bad Examples in Language Models [63.949883238901414]
  We present a unique angle of gradient analysis of loss functions that simultaneously reward good examples and penalize bad ones in LMs. We find that ExMATE serves as a superior surrogate for MLE, and that combining DPO with ExMATE instead of MLE further enhances both the statistical (5-7%) and generative (+18% win rate) performance.
  arXiv  Detail & Related papers  (2024-08-29T17:46:18Z)
• Quantifying Emergence in Large Language Models [31.608080868988825]
  We propose a quantifiable solution for estimating emergence of LLMs. Inspired by emergentism in dynamics, we quantify the strength of emergence by comparing the entropy reduction of the macroscopic (semantic) level with that of the microscopic (token) level. Our method demonstrates consistent behaviors across a suite of LMs under both in-context learning (ICL) and natural sentences.
  arXiv  Detail & Related papers  (2024-05-21T09:12:20Z)
• Statistical Agnostic Regression: a machine learning method to validate regression models [0.0]
  We introduce Statistical Agnostic Regression (SAR) for evaluating the statistical significance of machine learning (ML)-based linear regression models. We define a threshold that ensures there is sufficient evidence, with a probability of at least $1-eta$, to conclude the existence of a linear relationship in the population between the explanatory (feature) and the response (label) variables.
  arXiv  Detail & Related papers  (2024-02-23T09:19:26Z)
• Hyperparameter Tuning for Causal Inference with Double Machine Learning: A Simulation Study [4.526082390949313]
  We empirically assess the relationship between the predictive performance of machine learning methods and the resulting causal estimation. We conduct an extensive simulation study using data from the 2019 Atlantic Causal Inference Conference Data Challenge.
  arXiv  Detail & Related papers  (2024-02-07T09:01:51Z)
• Interpretable Causal Inference for Analyzing Wearable, Sensor, and Distributional Data [62.56890808004615]
  We develop an interpretable method for distributional data analysis that ensures trustworthy and robust decision-making. We demonstrate ADD MALTS' utility by studying the effectiveness of continuous glucose monitors in mitigating diabetes risks.
  arXiv  Detail & Related papers  (2023-12-17T00:42:42Z)
• To Repeat or Not To Repeat: Insights from Scaling LLM under Token-Crisis [50.31589712761807]
  Large language models (LLMs) are notoriously token-hungry during pre-training, and high-quality text data on the web is approaching its scaling limit for LLMs. We investigate the consequences of repeating pre-training data, revealing that the model is susceptible to overfitting. Second, we examine the key factors contributing to multi-epoch degradation, finding that significant factors include dataset size, model parameters, and training objectives.
  arXiv  Detail & Related papers  (2023-05-22T17:02:15Z)
• Proximal Causal Learning of Conditional Average Treatment Effects [0.0]
  We propose a tailored two-stage loss function for learning heterogeneous treatment effects. Our proposed estimator can be implemented by off-the-shelf loss-minimizing machine learning methods.
  arXiv  Detail & Related papers  (2023-01-26T02:56:36Z)
• Building Robust Machine Learning Models for Small Chemical Science Data: The Case of Shear Viscosity [3.4761212729163313]
  We train several Machine Learning models to predict the shear viscosity of a Lennard-Jones (LJ) fluid. Specifically, the issues related to model selection, performance estimation and uncertainty quantification were investigated.
  arXiv  Detail & Related papers  (2022-08-23T07:33:14Z)
• Provable Generalization of Overparameterized Meta-learning Trained with SGD [62.892930625034374]
  We study the generalization of a widely used meta-learning approach, Model-Agnostic Meta-Learning (MAML) We provide both upper and lower bounds for the excess risk of MAML, which captures how SGD dynamics affect these generalization bounds. Our theoretical findings are further validated by experiments.
  arXiv  Detail & Related papers  (2022-06-18T07:22:57Z)
• Comparison of meta-learners for estimating multi-valued treatment heterogeneous effects [2.294014185517203]
  Conditional Average Treatment Effects (CATE) estimation is one of the main challenges in causal inference with observational data. Nonparametric estimators called meta-learners have been developed to estimate the CATE with the main advantage of not restraining the estimation to a specific supervised learning method. This paper looks into meta-learners for estimating the heterogeneous effects of multi-valued treatments.
  arXiv  Detail & Related papers  (2022-05-29T16:46:21Z)
• Assessment of Treatment Effect Estimators for Heavy-Tailed Data [70.72363097550483]
  A central obstacle in the objective assessment of treatment effect (TE) estimators in randomized control trials (RCTs) is the lack of ground truth (or validation set) to test their performance. We provide a novel cross-validation-like methodology to address this challenge. We evaluate our methodology across 709 RCTs implemented in the Amazon supply chain.
  arXiv  Detail & Related papers  (2021-12-14T17:53:01Z)
• Causal Effect Variational Autoencoder with Uniform Treatment [50.895390968371665]
  Causal effect variational autoencoder (CEVAE) are trained to predict the outcome given observational treatment data. Uniform treatment variational autoencoders (UTVAE) are trained with uniform treatment distribution using importance sampling.
  arXiv  Detail & Related papers  (2021-11-16T17:40:57Z)
• SurvITE: Learning Heterogeneous Treatment Effects from Time-to-Event Data [83.50281440043241]
  We study the problem of inferring heterogeneous treatment effects from time-to-event data. We propose a novel deep learning method for treatment-specific hazard estimation based on balancing representations.
  arXiv  Detail & Related papers  (2021-10-26T20:13:17Z)
• Tight Mutual Information Estimation With Contrastive Fenchel-Legendre Optimization [69.07420650261649]
  We introduce a novel, simple, and powerful contrastive MI estimator named as FLO. Empirically, our FLO estimator overcomes the limitations of its predecessors and learns more efficiently. The utility of FLO is verified using an extensive set of benchmarks, which also reveals the trade-offs in practical MI estimation.
  arXiv  Detail & Related papers  (2021-07-02T15:20:41Z)
• Estimating Average Treatment Effects with Support Vector Machines [77.34726150561087]
  Support vector machine (SVM) is one of the most popular classification algorithms in the machine learning literature. We adapt SVM as a kernel-based weighting procedure that minimizes the maximum mean discrepancy between the treatment and control groups. We characterize the bias of causal effect estimation arising from this trade-off, connecting the proposed SVM procedure to the existing kernel balancing methods.
  arXiv  Detail & Related papers  (2021-02-23T20:22:56Z)
• Estimating heterogeneous survival treatment effect in observational data using machine learning [9.951103976634407]
  Methods for estimating heterogeneous treatment effect in observational data have largely focused on continuous or binary outcomes. Using flexible machine learning methods in the counterfactual framework is a promising approach to address challenges due to complex individual characteristics.
  arXiv  Detail & Related papers  (2020-08-17T01:02:14Z)
• Enabling Counterfactual Survival Analysis with Balanced Representations [64.17342727357618]
  Survival data are frequently encountered across diverse medical applications, i.e., drug development, risk profiling, and clinical trials. We propose a theoretically grounded unified framework for counterfactual inference applicable to survival outcomes.
  arXiv  Detail & Related papers  (2020-06-14T01:15:00Z)
• Localized Debiased Machine Learning: Efficient Inference on Quantile Treatment Effects and Beyond [69.83813153444115]
  We consider an efficient estimating equation for the (local) quantile treatment effect ((L)QTE) in causal inference. Debiased machine learning (DML) is a data-splitting approach to estimating high-dimensional nuisances. We propose localized debiased machine learning (LDML), which avoids this burdensome step.
  arXiv  Detail & Related papers  (2019-12-30T14:42:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.