Meta-Learners for Estimation of Causal Effects: Finite Sample Cross-Fit Performance

• URL: http://arxiv.org/abs/2201.12692v1
• Date: Sun, 30 Jan 2022 00:52:33 GMT
• Title: Meta-Learners for Estimation of Causal Effects: Finite Sample Cross-Fit Performance
• Authors: Gabriel Okasa
• Abstract summary: We study the finite sample performance of meta-learners for estimation of heterogeneous treatment effects. We find that the performance of the meta-learners in finite samples greatly depends on the estimation procedure.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Estimation of causal effects using machine learning methods has become an active research field in econometrics. In this paper, we study the finite sample performance of meta-learners for estimation of heterogeneous treatment effects under the usage of sample-splitting and cross-fitting to reduce the overfitting bias. In both synthetic and semi-synthetic simulations we find that the performance of the meta-learners in finite samples greatly depends on the estimation procedure. The results imply that sample-splitting and cross-fitting are beneficial in large samples for bias reduction and efficiency of the meta-learners, respectively, whereas full-sample estimation is preferable in small samples. Furthermore, we derive practical recommendations for application of specific meta-learners in empirical studies depending on particular data characteristics such as treatment shares and sample size.

Related papers

• Hierarchical Sparse Bayesian Multitask Model with Scalable Inference for Microbiome Analysis [1.361248247831476]
  This paper proposes a hierarchical Bayesian multitask learning model that is applicable to the general multi-task binary classification learning problem. We derive a computationally efficient inference algorithm based on variational inference to approximate the posterior distribution. We demonstrate the potential of the new approach on various synthetic datasets and for predicting human health status based on microbiome profile.
  arXiv  Detail & Related papers  (2025-02-04T18:23:22Z)
• A Gradient Analysis Framework for Rewarding Good and Penalizing Bad Examples in Language Models [63.949883238901414]
  We present a unique angle of gradient analysis of loss functions that simultaneously reward good examples and penalize bad ones in LMs. We find that ExMATE serves as a superior surrogate for MLE, and that combining DPO with ExMATE instead of MLE further enhances both the statistical (5-7%) and generative (+18% win rate) performance.
  arXiv  Detail & Related papers  (2024-08-29T17:46:18Z)
• Estimating Distributional Treatment Effects in Randomized Experiments: Machine Learning for Variance Reduction [6.909352249236339]
  We propose a novel regression adjustment method designed for estimating distributional treatment effect parameters in randomized experiments. Our approach incorporates pre-treatment co-treatments into a distributional regression framework, utilizing machine learning techniques to improve the precision of distributional treatment effect estimators.
  arXiv  Detail & Related papers  (2024-07-22T20:28:29Z)
• Data Pruning via Moving-one-Sample-out [61.45441981346064]
  We propose a novel data-pruning approach called moving-one-sample-out (MoSo) MoSo aims to identify and remove the least informative samples from the training set. Experimental results demonstrate that MoSo effectively mitigates severe performance degradation at high pruning ratios.
  arXiv  Detail & Related papers  (2023-10-23T08:00:03Z)
• Boosting Differentiable Causal Discovery via Adaptive Sample Reweighting [62.23057729112182]
  Differentiable score-based causal discovery methods learn a directed acyclic graph from observational data. We propose a model-agnostic framework to boost causal discovery performance by dynamically learning the adaptive weights for the Reweighted Score function, ReScore.
  arXiv  Detail & Related papers  (2023-03-06T14:49:59Z)
• Evaluating Synthetic Tabular Data Generated To Augment Small Sample Datasets [0.0]
  This work proposes a method to evaluate synthetic data generated to augment small sample datasets. Our experiments reveal significant inconsistencies between global metrics and topological measures. No single metric reliably captures both distributional and structural similarity.
  arXiv  Detail & Related papers  (2022-11-19T18:18:52Z)
• MetaRF: Differentiable Random Forest for Reaction Yield Prediction with a Few Trails [58.47364143304643]
  In this paper, we focus on the reaction yield prediction problem. We first put forth MetaRF, an attention-based differentiable random forest model specially designed for the few-shot yield prediction. To improve the few-shot learning performance, we further introduce a dimension-reduction based sampling method.
  arXiv  Detail & Related papers  (2022-08-22T06:40:13Z)
• Rethinking Collaborative Metric Learning: Toward an Efficient Alternative without Negative Sampling [156.7248383178991]
  Collaborative Metric Learning (CML) paradigm has aroused wide interest in the area of recommendation systems (RS) We find that negative sampling would lead to a biased estimation of the generalization error. Motivated by this, we propose an efficient alternative without negative sampling for CML named textitSampling-Free Collaborative Metric Learning (SFCML)
  arXiv  Detail & Related papers  (2022-06-23T08:50:22Z)
• Comparison of meta-learners for estimating multi-valued treatment heterogeneous effects [2.294014185517203]
  Conditional Average Treatment Effects (CATE) estimation is one of the main challenges in causal inference with observational data. Nonparametric estimators called meta-learners have been developed to estimate the CATE with the main advantage of not restraining the estimation to a specific supervised learning method. This paper looks into meta-learners for estimating the heterogeneous effects of multi-valued treatments.
  arXiv  Detail & Related papers  (2022-05-29T16:46:21Z)
• Rethinking InfoNCE: How Many Negative Samples Do You Need? [54.146208195806636]
  We study how many negative samples are optimal for InfoNCE in different scenarios via a semi-quantitative theoretical framework. We estimate the optimal negative sampling ratio using the $K$ value that maximizes the training effectiveness function.
  arXiv  Detail & Related papers  (2021-05-27T08:38:29Z)
• Weighting-Based Treatment Effect Estimation via Distribution Learning [14.438302755258547]
  We develop a distribution learning-based weighting method for treatment effect estimation. Our method outperforms several cutting-edge weighting-only benchmarking methods. It maintains its advantage under a doubly-robust estimation framework.
  arXiv  Detail & Related papers  (2020-12-26T20:15:44Z)
• Double machine learning for sample selection models [0.12891210250935145]
  This paper considers the evaluation of discretely distributed treatments when outcomes are only observed for a subpopulation due to sample selection or outcome attrition. We make use of (a) Neyman-orthogonal, doubly robust, and efficient score functions, which imply the robustness of treatment effect estimation to moderate regularization biases in the machine learning-based estimation of the outcome, treatment, or sample selection models and (b) sample splitting (or cross-fitting) to prevent overfitting bias.
  arXiv  Detail & Related papers  (2020-11-30T19:40:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.