Improving the Finite Sample Estimation of Average Treatment Effects using Double/Debiased Machine Learning with Propensity Score Calibration

• URL: http://arxiv.org/abs/2409.04874v2
• Date: Thu, 16 Jan 2025 13:05:14 GMT
• Title: Improving the Finite Sample Estimation of Average Treatment Effects using Double/Debiased Machine Learning with Propensity Score Calibration
• Authors: Daniele Ballinari, Nora Bearth,
• Abstract summary: This paper investigates the use of probability calibration approaches within the Double/debiased machine learning framework. We show that calibrating propensity scores may significantly reduce the root mean squared error of DML estimates. We showcase it in an empirical example and provide conditions under which calibration does not alter the properties of the DML estimator.
• Score: 0.0
• License:
• Abstract: In the last decade, machine learning techniques have gained popularity for estimating causal effects. One machine learning approach that can be used for estimating an average treatment effect is Double/debiased machine learning (DML) (Chernozhukov et al., 2018). This approach uses a double-robust score function that relies on the prediction of nuisance functions, such as the propensity score, which is the probability of treatment assignment conditional on covariates. Estimators relying on double-robust score functions are highly sensitive to errors in propensity score predictions. Machine learners increase the severity of this problem as they tend to over- or underestimate these probabilities. Several calibration approaches have been proposed to improve probabilistic forecasts of machine learners. This paper investigates the use of probability calibration approaches within the DML framework. Simulation results demonstrate that calibrating propensity scores may significantly reduces the root mean squared error of DML estimates of the average treatment effect in finite samples. We showcase it in an empirical example and provide conditions under which calibration does not alter the asymptotic properties of the DML estimator.

Related papers

• Automatic doubly robust inference for linear functionals via calibrated debiased machine learning [0.9694940903078658]
  We propose a debiased machine learning estimator for doubly robust inference. A C-DML estimator maintains linearity when either the outcome regression or the Riesz representer of the linear functional is estimated sufficiently well. Our theoretical and empirical results support the use of C-DML to mitigate bias arising from the inconsistent or slow estimation of nuisance functions.
  arXiv  Detail & Related papers  (2024-11-05T03:32:30Z)
• Semiparametric conformal prediction [79.6147286161434]
  Risk-sensitive applications require well-calibrated prediction sets over multiple, potentially correlated target variables. We treat the scores as random vectors and aim to construct the prediction set accounting for their joint correlation structure. We report desired coverage and competitive efficiency on a range of real-world regression problems.
  arXiv  Detail & Related papers  (2024-11-04T14:29:02Z)
• Calibrating doubly-robust estimators with unbalanced treatment assignment [0.0]
  We propose a simple extension of the DML estimator which undersamples data for propensity score modeling. The paper provides theoretical results showing that the estimator retains the estimator's properties and calibrates scores to match the original distribution.
  arXiv  Detail & Related papers  (2024-03-03T18:40:11Z)
• Calibrated and Conformal Propensity Scores for Causal Effect Estimation [10.209143402485406]
  We argue that the probabilistic output of a learned propensity score model should be calibrated. Calibrated propensity scores improve the speed of GWAS analysis by more than two-fold.
  arXiv  Detail & Related papers  (2023-06-01T06:26:26Z)
• Monotonicity and Double Descent in Uncertainty Estimation with Gaussian Processes [52.92110730286403]
  It is commonly believed that the marginal likelihood should be reminiscent of cross-validation metrics and that both should deteriorate with larger input dimensions. We prove that by tuning hyper parameters, the performance, as measured by the marginal likelihood, improves monotonically with the input dimension. We also prove that cross-validation metrics exhibit qualitatively different behavior that is characteristic of double descent.
  arXiv  Detail & Related papers  (2022-10-14T08:09:33Z)
• Imputation-Free Learning from Incomplete Observations [73.15386629370111]
  We introduce the importance of guided gradient descent (IGSGD) method to train inference from inputs containing missing values without imputation. We employ reinforcement learning (RL) to adjust the gradients used to train the models via back-propagation. Our imputation-free predictions outperform the traditional two-step imputation-based predictions using state-of-the-art imputation methods.
  arXiv  Detail & Related papers  (2021-07-05T12:44:39Z)
• Machine Learning for Variance Reduction in Online Experiments [1.9181913148426697]
  We propose a machine learning regression-adjusted treatment effect estimator, which we call MLRATE. MLRATE uses machine learning predictors of the outcome to reduce estimator variance. In A/A tests, for a set of 48 outcome metrics commonly monitored in Facebook experiments, the estimator has over 70% lower variance than the simple difference-in-means estimator.
  arXiv  Detail & Related papers  (2021-06-14T09:35:54Z)
• Automatic Double Machine Learning for Continuous Treatment Effects [0.0]
  We introduce and prove normality for a new nonparametric estimator of continuous treatment effects. We estimate the average dose-response function - the expected value of an outcome of interest at a particular level of the treatment level.
  arXiv  Detail & Related papers  (2021-04-21T03:17:40Z)
• Scalable Marginal Likelihood Estimation for Model Selection in Deep Learning [78.83598532168256]
  Marginal-likelihood based model-selection is rarely used in deep learning due to estimation difficulties. Our work shows that marginal likelihoods can improve generalization and be useful when validation data is unavailable.
  arXiv  Detail & Related papers  (2021-04-11T09:50:24Z)
• Machine learning for causal inference: on the use of cross-fit estimators [77.34726150561087]
  Doubly-robust cross-fit estimators have been proposed to yield better statistical properties. We conducted a simulation study to assess the performance of several estimators for the average causal effect (ACE) When used with machine learning, the doubly-robust cross-fit estimators substantially outperformed all of the other estimators in terms of bias, variance, and confidence interval coverage.
  arXiv  Detail & Related papers  (2020-04-21T23:09:55Z)
• Localized Debiased Machine Learning: Efficient Inference on Quantile Treatment Effects and Beyond [69.83813153444115]
  We consider an efficient estimating equation for the (local) quantile treatment effect ((L)QTE) in causal inference. Debiased machine learning (DML) is a data-splitting approach to estimating high-dimensional nuisances. We propose localized debiased machine learning (LDML), which avoids this burdensome step.
  arXiv  Detail & Related papers  (2019-12-30T14:42:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.