Amortized Inference for Causal Structure Learning
- URL: http://arxiv.org/abs/2205.12934v1
- Date: Wed, 25 May 2022 17:37:08 GMT
- Title: Amortized Inference for Causal Structure Learning
- Authors: Lars Lorch, Scott Sussex, Jonas Rothfuss, Andreas Krause, Bernhard
Sch\"olkopf
- Abstract summary: Learning causal structure poses a search problem that typically involves evaluating structures using a score or independence test.
We train a variational inference model to predict the causal structure from observational/interventional data.
Our models exhibit robust generalization capabilities under substantial distribution shift.
- Score: 72.84105256353801
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Learning causal structure poses a combinatorial search problem that typically
involves evaluating structures using a score or independence test. The
resulting search is costly, and designing suitable scores or tests that capture
prior knowledge is difficult. In this work, we propose to amortize the process
of causal structure learning. Rather than searching over causal structures
directly, we train a variational inference model to predict the causal
structure from observational/interventional data. Our inference model acquires
domain-specific inductive bias for causal discovery solely from data generated
by a simulator. This allows us to bypass both the search over graphs and the
hand-engineering of suitable score functions. Moreover, the architecture of our
inference model is permutation invariant w.r.t. the data points and permutation
equivariant w.r.t. the variables, facilitating generalization to significantly
larger problem instances than seen during training. On synthetic data and
semi-synthetic gene expression data, our models exhibit robust generalization
capabilities under substantial distribution shift and significantly outperform
existing algorithms, especially in the challenging genomics domain.
Related papers
- Learning Divergence Fields for Shift-Robust Graph Representations [73.11818515795761]
In this work, we propose a geometric diffusion model with learnable divergence fields for the challenging problem with interdependent data.
We derive a new learning objective through causal inference, which can guide the model to learn generalizable patterns of interdependence that are insensitive across domains.
arXiv Detail & Related papers (2024-06-07T14:29:21Z) - FiP: a Fixed-Point Approach for Causal Generative Modeling [20.88890689294816]
We propose a new and equivalent formalism that does not require DAGs to describe fixed-point problems on the causally ordered variables.
We show three important cases where they can be uniquely recovered given the topological ordering (TO)
arXiv Detail & Related papers (2024-04-10T12:29:05Z) - Sample, estimate, aggregate: A recipe for causal discovery foundation models [28.116832159265964]
We train a supervised model that learns to predict a larger causal graph from the outputs of classical causal discovery algorithms run over subsets of variables.
Our approach is enabled by the observation that typical errors in the outputs of classical methods remain comparable across datasets.
Experiments on real and synthetic data demonstrate that this model maintains high accuracy in the face of misspecification or distribution shift.
arXiv Detail & Related papers (2024-02-02T21:57:58Z) - Learning to Bound Counterfactual Inference in Structural Causal Models
from Observational and Randomised Data [64.96984404868411]
We derive a likelihood characterisation for the overall data that leads us to extend a previous EM-based algorithm.
The new algorithm learns to approximate the (unidentifiability) region of model parameters from such mixed data sources.
It delivers interval approximations to counterfactual results, which collapse to points in the identifiable case.
arXiv Detail & Related papers (2022-12-06T12:42:11Z) - Learning Neural Causal Models with Active Interventions [83.44636110899742]
We introduce an active intervention-targeting mechanism which enables a quick identification of the underlying causal structure of the data-generating process.
Our method significantly reduces the required number of interactions compared with random intervention targeting.
We demonstrate superior performance on multiple benchmarks from simulated to real-world data.
arXiv Detail & Related papers (2021-09-06T13:10:37Z) - Systematic Evaluation of Causal Discovery in Visual Model Based
Reinforcement Learning [76.00395335702572]
A central goal for AI and causality is the joint discovery of abstract representations and causal structure.
Existing environments for studying causal induction are poorly suited for this objective because they have complicated task-specific causal graphs.
In this work, our goal is to facilitate research in learning representations of high-level variables as well as causal structures among them.
arXiv Detail & Related papers (2021-07-02T05:44:56Z) - Learning Curves for SGD on Structured Features [23.40229188549055]
We show that the geometry of the data in the induced feature space is crucial to accurately predict the test error throughout learning.
We show that modeling the geometry of the data in the induced feature space is indeed crucial to accurately predict the test error throughout learning.
arXiv Detail & Related papers (2021-06-04T20:48:20Z) - Parsimonious Inference [0.0]
Parsimonious inference is an information-theoretic formulation of inference over arbitrary architectures.
Our approaches combine efficient encodings with prudent sampling strategies to construct predictive ensembles without cross-validation.
arXiv Detail & Related papers (2021-03-03T04:13:14Z) - Learning Causal Models Online [103.87959747047158]
Predictive models can rely on spurious correlations in the data for making predictions.
One solution for achieving strong generalization is to incorporate causal structures in the models.
We propose an online algorithm that continually detects and removes spurious features.
arXiv Detail & Related papers (2020-06-12T20:49:20Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.