Treatment Effect Risk: Bounds and Inference

• URL: http://arxiv.org/abs/2201.05893v1
• Date: Sat, 15 Jan 2022 17:21:26 GMT
• Title: Treatment Effect Risk: Bounds and Inference
• Authors: Nathan Kallus
• Abstract summary: Since the average treatment effect measures the change in social welfare, even if positive, there is a risk of negative effect on, say, some 10% of the population. In this paper we consider how to nonetheless assess this important risk measure, formalized as the conditional value at risk (CVaR) of the ITE distribution. Some bounds can also be interpreted as summarizing a complex CATE function into a single metric and are of interest independently of being a bound.
• Score: 58.442274475425144
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Since the average treatment effect (ATE) measures the change in social welfare, even if positive, there is a risk of negative effect on, say, some 10% of the population. Assessing such risk is difficult, however, because any one individual treatment effect (ITE) is never observed so the 10% worst-affected cannot be identified, while distributional treatment effects only compare the first deciles within each treatment group, which does not correspond to any 10%-subpopulation. In this paper we consider how to nonetheless assess this important risk measure, formalized as the conditional value at risk (CVaR) of the ITE distribution. We leverage the availability of pre-treatment covariates and characterize the tightest-possible upper and lower bounds on ITE-CVaR given by the covariate-conditional average treatment effect (CATE) function. Some bounds can also be interpreted as summarizing a complex CATE function into a single metric and are of interest independently of being a bound. We then proceed to study how to estimate these bounds efficiently from data and construct confidence intervals. This is challenging even in randomized experiments as it requires understanding the distribution of the unknown CATE function, which can be very complex if we use rich covariates so as to best control for heterogeneity. We develop a debiasing method that overcomes this and prove it enjoys favorable statistical properties even when CATE and other nuisances are estimated by black-box machine learning or even inconsistently. Studying a hypothetical change to French job-search counseling services, our bounds and inference demonstrate a small social benefit entails a negative impact on a substantial subpopulation.

Related papers

• Quantifying Aleatoric Uncertainty of the Treatment Effect: A Novel Orthogonal Learner [72.20769640318969]
  Estimating causal quantities from observational data is crucial for understanding the safety and effectiveness of medical treatments. Medical practitioners require not only estimating averaged causal quantities, but also understanding the randomness of the treatment effect as a random variable. This randomness is referred to as aleatoric uncertainty and is necessary for understanding the probability of benefit from treatment or quantiles of the treatment effect.
  arXiv  Detail & Related papers  (2024-11-05T18:14:49Z)
• Accounting for Missing Covariates in Heterogeneous Treatment Estimation [17.09751619857397]
  We introduce a novel partial identification strategy based on ideas from ecological inference. We show that our framework can produce bounds that are much tighter than would otherwise be possible.
  arXiv  Detail & Related papers  (2024-10-21T05:47:07Z)
• B-Learner: Quasi-Oracle Bounds on Heterogeneous Causal Effects Under Hidden Confounding [51.74479522965712]
  We propose a meta-learner called the B-Learner, which can efficiently learn sharp bounds on the CATE function under limits on hidden confounding. We prove its estimates are valid, sharp, efficient, and have a quasi-oracle property with respect to the constituent estimators under more general conditions than existing methods.
  arXiv  Detail & Related papers  (2023-04-20T18:07:19Z)
• Proximal Causal Learning of Conditional Average Treatment Effects [0.0]
  We propose a tailored two-stage loss function for learning heterogeneous treatment effects. Our proposed estimator can be implemented by off-the-shelf loss-minimizing machine learning methods.
  arXiv  Detail & Related papers  (2023-01-26T02:56:36Z)
• Robust and Agnostic Learning of Conditional Distributional Treatment Effects [62.44901952244514]
  The conditional average treatment effect (CATE) is the best point prediction of individual causal effects. In aggregate analyses, this is usually addressed by measuring distributional treatment effect (DTE) We provide a new robust and model-agnostic methodology for learning the conditional DTE (CDTE) for a wide class of problems.
  arXiv  Detail & Related papers  (2022-05-23T17:40:31Z)
• What's the Harm? Sharp Bounds on the Fraction Negatively Affected by Treatment [58.442274475425144]
  We develop a robust inference algorithm that is efficient almost regardless of how and how fast these functions are learned. We demonstrate our method in simulation studies and in a case study of career counseling for the unemployed.
  arXiv  Detail & Related papers  (2022-05-20T17:36:33Z)
• Assessment of Treatment Effect Estimators for Heavy-Tailed Data [70.72363097550483]
  A central obstacle in the objective assessment of treatment effect (TE) estimators in randomized control trials (RCTs) is the lack of ground truth (or validation set) to test their performance. We provide a novel cross-validation-like methodology to address this challenge. We evaluate our methodology across 709 RCTs implemented in the Amazon supply chain.
  arXiv  Detail & Related papers  (2021-12-14T17:53:01Z)
• Conformal Inference of Counterfactuals and Individual Treatment Effects [6.810856082577402]
  We propose a conformal inference-based approach that can produce reliable interval estimates for counterfactuals and individual treatment effects. Existing methods suffer from a significant coverage deficit even in simple models.
  arXiv  Detail & Related papers  (2020-06-11T01:03:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.