Invariant Representation Learning for Treatment Effect Estimation

• URL: http://arxiv.org/abs/2011.12379v2
• Date: Tue, 27 Jul 2021 06:45:08 GMT
• Title: Invariant Representation Learning for Treatment Effect Estimation
• Authors: Claudia Shi, Victor Veitch, David Blei
• Abstract summary: We develop Nearly Invariant Causal Estimation (NICE) NICE uses invariant risk minimization (IRM) [Arj19] to learn a representation of the covariates that, under some assumptions, strips out bad controls but preserves sufficient information to adjust for confounding. We evaluate NICE on both synthetic and semi-synthetic data.
• Score: 12.269209356327526
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The defining challenge for causal inference from observational data is the presence of `confounders', covariates that affect both treatment assignment and the outcome. To address this challenge, practitioners collect and adjust for the covariates, hoping that they adequately correct for confounding. However, including every observed covariate in the adjustment runs the risk of including `bad controls', variables that induce bias when they are conditioned on. The problem is that we do not always know which variables in the covariate set are safe to adjust for and which are not. To address this problem, we develop Nearly Invariant Causal Estimation (NICE). NICE uses invariant risk minimization (IRM) [Arj19] to learn a representation of the covariates that, under some assumptions, strips out bad controls but preserves sufficient information to adjust for confounding. Adjusting for the learned representation, rather than the covariates themselves, avoids the induced bias and provides valid causal inferences. We evaluate NICE on both synthetic and semi-synthetic data. When the covariates contain unknown collider variables and other bad controls, NICE performs better than adjusting for all the covariates.

Related papers

• Representation Learning Preserving Ignorability and Covariate Matching for Treatment Effects [18.60804431844023]
  Estimating treatment effects from observational data is challenging due to hidden confounding. A common framework to address both hidden confounding and selection bias is missing.
  arXiv  Detail & Related papers  (2025-04-29T09:33:56Z)
• Towards Self-Supervised Covariance Estimation in Deep Heteroscedastic Regression [102.24287051757469]
  We study self-supervised covariance estimation in deep heteroscedastic regression. We derive an upper bound on the 2-Wasserstein distance between normal distributions. Experiments over a wide range of synthetic and real datasets demonstrate that the proposed 2-Wasserstein bound coupled with pseudo label annotations results in a computationally cheaper yet accurate deep heteroscedastic regression.
  arXiv  Detail & Related papers  (2025-02-14T22:37:11Z)
• Local Learning for Covariate Selection in Nonparametric Causal Effect Estimation with Latent Variables [13.12743473333296]
  Estimating causal effects from nonexperimental data is a fundamental problem in many fields of science. We propose a novel local learning approach for covariate selection in nonparametric causal effect estimation. We validate our algorithm through extensive experiments on both synthetic and real-world data.
  arXiv  Detail & Related papers  (2024-11-25T12:08:54Z)
• Causal Inference from Text: Unveiling Interactions between Variables [20.677407402398405]
  Existing methods only account for confounding covariables that affect both treatment and outcome. This bias arises from insufficient consideration of non-confounding covariables. In this work, we aim to mitigate the bias by unveiling interactions between different variables.
  arXiv  Detail & Related papers  (2023-11-09T11:29:44Z)
• TIC-TAC: A Framework for Improved Covariance Estimation in Deep Heteroscedastic Regression [109.69084997173196]
  Deepscedastic regression involves jointly optimizing the mean and covariance of the predicted distribution using the negative log-likelihood. Recent works show that this may result in sub-optimal convergence due to the challenges associated with covariance estimation. We study two questions: (1) Does the predicted covariance truly capture the randomness of the predicted mean? Our results show that not only does TIC accurately learn the covariance, it additionally facilitates an improved convergence of the negative log-likelihood.
  arXiv  Detail & Related papers  (2023-10-29T09:54:03Z)
• Robust Learning via Conditional Prevalence Adjustment [7.480241867887245]
  Deep learning models might fail catastrophically in unseen sites. We propose a method called CoPA (Conditional Prevalence-Adjustment) for anti-causal tasks.
  arXiv  Detail & Related papers  (2023-10-24T12:13:49Z)
• Orthogonal SVD Covariance Conditioning and Latent Disentanglement [65.67315418971688]
  Inserting an SVD meta-layer into neural networks is prone to make the covariance ill-conditioned. We propose Nearest Orthogonal Gradient (NOG) and Optimal Learning Rate (OLR) Experiments on visual recognition demonstrate that our methods can simultaneously improve covariance conditioning and generalization.
  arXiv  Detail & Related papers  (2022-12-11T20:31:31Z)
• Stable Learning via Sparse Variable Independence [41.632242102167844]
  We propose SVI (Sparse Variable Independence) for the covariate-shift generalization problem. We introduce sparsity constraint to compensate for the imperfectness of sample reweighting under the finite-sample setting. Experiments on both synthetic and real-world datasets demonstrate the improvement of SVI.
  arXiv  Detail & Related papers  (2022-12-02T05:59:30Z)
• Partial Identification with Noisy Covariates: A Robust Optimization Approach [94.10051154390237]
  Causal inference from observational datasets often relies on measuring and adjusting for covariates. We show that this robust optimization approach can extend a wide range of causal adjustment methods to perform partial identification. Across synthetic and real datasets, we find that this approach provides ATE bounds with a higher coverage probability than existing methods.
  arXiv  Detail & Related papers  (2022-02-22T04:24:26Z)
• Conditional Contrastive Learning with Kernel [107.5989144369343]
  Conditional Contrastive Learning with Kernel (CCL-K) This paper presents Conditional Contrastive Learning with Kernel that converts existing conditional contrastive objectives into alternative forms that mitigate the insufficient data problem. We conduct experiments using weakly supervised, fair, and hard negatives contrastive learning, showing CCL-K outperforms state-of-the-art baselines.
  arXiv  Detail & Related papers  (2022-02-11T05:37:54Z)
• Controlling for multiple covariates [0.0]
  A fundamental problem in statistics is to compare the outcomes attained by members of subpopulations. Comparison makes the most sense when performed separately for individuals who are similar according to certain characteristics.
  arXiv  Detail & Related papers  (2021-12-01T17:37:36Z)
• Deconfounding Scores: Feature Representations for Causal Effect Estimation with Weak Overlap [140.98628848491146]
  We introduce deconfounding scores, which induce better overlap without biasing the target of estimation. We show that deconfounding scores satisfy a zero-covariance condition that is identifiable in observed data. In particular, we show that this technique could be an attractive alternative to standard regularizations.
  arXiv  Detail & Related papers  (2021-04-12T18:50:11Z)
• Evaluating Prediction-Time Batch Normalization for Robustness under Covariate Shift [81.74795324629712]
  We call prediction-time batch normalization, which significantly improves model accuracy and calibration under covariate shift. We show that prediction-time batch normalization provides complementary benefits to existing state-of-the-art approaches for improving robustness. The method has mixed results when used alongside pre-training, and does not seem to perform as well under more natural types of dataset shift.
  arXiv  Detail & Related papers  (2020-06-19T05:08:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.