ACTIVA: Amortized Causal Effect Estimation without Graphs via Transformer-based Variational Autoencoder

• URL: http://arxiv.org/abs/2503.01290v1
• Date: Mon, 03 Mar 2025 08:28:25 GMT
• Title: ACTIVA: Amortized Causal Effect Estimation without Graphs via Transformer-based Variational Autoencoder
• Authors: Andreas Sauter, Saber Salehkaleybar, Aske Plaat, Erman Acar,
• Abstract summary: We propose a novel conditional variational autoencoder architecture, named ACTIVA, that extends causal transformer encoders to predict causal effects as mixtures of Gaussians.<n>Our method requires no causal graph and predicts interventional distributions given only observational data and a queried intervention.<n>By amortizing over many simulated instances, it enables zero-shot generalization to novel datasets without retraining.
• Score: 7.987204219322316
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Predicting the distribution of outcomes under hypothetical interventions is crucial in domains like healthcare, economics, and policy-making. Current methods often rely on strong assumptions, such as known causal graphs or parametric models, and lack amortization across problem instances, limiting their practicality. We propose a novel transformer-based conditional variational autoencoder architecture, named ACTIVA, that extends causal transformer encoders to predict causal effects as mixtures of Gaussians. Our method requires no causal graph and predicts interventional distributions given only observational data and a queried intervention. By amortizing over many simulated instances, it enables zero-shot generalization to novel datasets without retraining. Experiments demonstrate accurate predictions for synthetic and semi-synthetic data, showcasing the effectiveness of our graph-free, amortized causal inference approach.

Related papers

• Generative Intervention Models for Causal Perturbation Modeling [80.72074987374141]
  In many applications, it is a priori unknown which mechanisms of a system are modified by an external perturbation. We propose a generative intervention model (GIM) that learns to map these perturbation features to distributions over atomic interventions.
  arXiv  Detail & Related papers  (2024-11-21T10:37:57Z)
• Sample, estimate, aggregate: A recipe for causal discovery foundation models [28.116832159265964]
  We train a supervised model that learns to predict a larger causal graph from the outputs of classical causal discovery algorithms run over subsets of variables. Our approach is enabled by the observation that typical errors in the outputs of classical methods remain comparable across datasets. Experiments on real and synthetic data demonstrate that this model maintains high accuracy in the face of misspecification or distribution shift.
  arXiv  Detail & Related papers  (2024-02-02T21:57:58Z)
• Advancing Counterfactual Inference through Nonlinear Quantile Regression [77.28323341329461]
  We propose a framework for efficient and effective counterfactual inference implemented with neural networks. The proposed approach enhances the capacity to generalize estimated counterfactual outcomes to unseen data. Empirical results conducted on multiple datasets offer compelling support for our theoretical assertions.
  arXiv  Detail & Related papers  (2023-06-09T08:30:51Z)
• Nonparametric Identifiability of Causal Representations from Unknown Interventions [63.1354734978244]
  We study causal representation learning, the task of inferring latent causal variables and their causal relations from mixtures of the variables. Our goal is to identify both the ground truth latents and their causal graph up to a set of ambiguities which we show to be irresolvable from interventional data.
  arXiv  Detail & Related papers  (2023-06-01T10:51:58Z)
• Data-Driven Influence Functions for Optimization-Based Causal Inference [105.5385525290466]
  We study a constructive algorithm that approximates Gateaux derivatives for statistical functionals by finite differencing. We study the case where probability distributions are not known a priori but need to be estimated from data.
  arXiv  Detail & Related papers  (2022-08-29T16:16:22Z)
• Neuro-Symbolic Entropy Regularization [78.16196949641079]
  In structured prediction, the goal is to jointly predict many output variables that together encode a structured object. One approach -- entropy regularization -- posits that decision boundaries should lie in low-probability regions. We propose a loss, neuro-symbolic entropy regularization, that encourages the model to confidently predict a valid object.
  arXiv  Detail & Related papers  (2022-01-25T06:23:10Z)
• Estimation of Bivariate Structural Causal Models by Variational Gaussian Process Regression Under Likelihoods Parametrised by Normalising Flows [74.85071867225533]
  Causal mechanisms can be described by structural causal models. One major drawback of state-of-the-art artificial intelligence is its lack of explainability.
  arXiv  Detail & Related papers  (2021-09-06T14:52:58Z)
• Which Invariance Should We Transfer? A Causal Minimax Learning Approach [18.71316951734806]
  We present a comprehensive minimax analysis from a causal perspective. We propose an efficient algorithm to search for the subset with minimal worst-case risk. The effectiveness and efficiency of our methods are demonstrated on synthetic data and the diagnosis of Alzheimer's disease.
  arXiv  Detail & Related papers  (2021-07-05T09:07:29Z)
• Efficient Causal Inference from Combined Observational and Interventional Data through Causal Reductions [68.6505592770171]
  Unobserved confounding is one of the main challenges when estimating causal effects. We propose a novel causal reduction method that replaces an arbitrary number of possibly high-dimensional latent confounders. We propose a learning algorithm to estimate the parameterized reduced model jointly from observational and interventional data.
  arXiv  Detail & Related papers  (2021-03-08T14:29:07Z)
• Identification of Latent Variables From Graphical Model Residuals [0.0]
  We present a novel method to control for the latent space when estimating a DAG by iteratively deriving proxies for the latent space from the residuals of the inferred model. We show that any improvement of prediction of an outcome is intrinsically capped and cannot rise beyond a certain limit as compared to the confounded model.
  arXiv  Detail & Related papers  (2021-01-07T02:28:49Z)
• Causal Autoregressive Flows [4.731404257629232]
  We highlight an intrinsic correspondence between a simple family of autoregressive normalizing flows and identifiable causal models. We exploit the fact that autoregressive flow architectures define an ordering over variables, analogous to a causal ordering, to show that they are well-suited to performing a range of causal inference tasks.
  arXiv  Detail & Related papers  (2020-11-04T13:17:35Z)
• Autoregressive flow-based causal discovery and inference [4.83420384410068]
  Autoregressive flow models are well-suited to performing a range of causal inference tasks. We exploit the fact that autoregressive architectures define an ordering over variables, analogous to a causal ordering. We present examples over synthetic data where autoregressive flows, when trained under the correct causal ordering, are able to make accurate interventional and counterfactual predictions.
  arXiv  Detail & Related papers  (2020-07-18T10:02:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.