BayesDAG: Gradient-Based Posterior Inference for Causal Discovery
- URL: http://arxiv.org/abs/2307.13917v2
- Date: Fri, 8 Dec 2023 15:49:49 GMT
- Title: BayesDAG: Gradient-Based Posterior Inference for Causal Discovery
- Authors: Yashas Annadani, Nick Pawlowski, Joel Jennings, Stefan Bauer, Cheng
Zhang, Wenbo Gong
- Abstract summary: We introduce a scalable causal discovery framework based on a combination of Markov Chain Monte Carlo and Variational Inference.
Our approach directly samples DAGs from the posterior without requiring any DAG regularization.
We derive a novel equivalence to the permutation-based DAG learning, which opens up possibilities of using any relaxed estimator defined over permutations.
- Score: 30.027520859604955
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Bayesian causal discovery aims to infer the posterior distribution over
causal models from observed data, quantifying epistemic uncertainty and
benefiting downstream tasks. However, computational challenges arise due to
joint inference over combinatorial space of Directed Acyclic Graphs (DAGs) and
nonlinear functions. Despite recent progress towards efficient posterior
inference over DAGs, existing methods are either limited to variational
inference on node permutation matrices for linear causal models, leading to
compromised inference accuracy, or continuous relaxation of adjacency matrices
constrained by a DAG regularizer, which cannot ensure resulting graphs are
DAGs. In this work, we introduce a scalable Bayesian causal discovery framework
based on a combination of stochastic gradient Markov Chain Monte Carlo
(SG-MCMC) and Variational Inference (VI) that overcomes these limitations. Our
approach directly samples DAGs from the posterior without requiring any DAG
regularization, simultaneously draws function parameter samples and is
applicable to both linear and nonlinear causal models. To enable our approach,
we derive a novel equivalence to the permutation-based DAG learning, which
opens up possibilities of using any relaxed gradient estimator defined over
permutations. To our knowledge, this is the first framework applying
gradient-based MCMC sampling for causal discovery. Empirical evaluation on
synthetic and real-world datasets demonstrate our approach's effectiveness
compared to state-of-the-art baselines.
Related papers
- Induced Covariance for Causal Discovery in Linear Sparse Structures [55.2480439325792]
Causal models seek to unravel the cause-effect relationships among variables from observed data.
This paper introduces a novel causal discovery algorithm designed for settings in which variables exhibit linearly sparse relationships.
arXiv Detail & Related papers (2024-10-02T04:01:38Z) - Scalable Variational Causal Discovery Unconstrained by Acyclicity [6.954510776782872]
We propose a scalable Bayesian approach to learn the posterior distribution over causal graphs given observational data.
We introduce a novel differentiable DAG sampling method that can generate a valid acyclic causal graph.
We are able to model the posterior distribution over causal graphs using a simple variational distribution over a continuous domain.
arXiv Detail & Related papers (2024-07-06T07:56:23Z) - ProDAG: Projection-Induced Variational Inference for Directed Acyclic Graphs [8.556906995059324]
Directed acyclic graph (DAG) learning is a rapidly expanding field of research.
It remains statistically and computationally challenging to learn a single (point estimate) DAG from data, let alone provide uncertainty quantification.
Our article addresses the difficult task of quantifying graph uncertainty by developing a Bayesian variational inference framework based on novel distributions that have support directly on the space of DAGs.
arXiv Detail & Related papers (2024-05-24T03:04:28Z) - Learning Linear Causal Representations from Interventions under General
Nonlinear Mixing [52.66151568785088]
We prove strong identifiability results given unknown single-node interventions without access to the intervention targets.
This is the first instance of causal identifiability from non-paired interventions for deep neural network embeddings.
arXiv Detail & Related papers (2023-06-04T02:32:12Z) - GibbsDDRM: A Partially Collapsed Gibbs Sampler for Solving Blind Inverse
Problems with Denoising Diffusion Restoration [64.8770356696056]
We propose GibbsDDRM, an extension of Denoising Diffusion Restoration Models (DDRM) to a blind setting in which the linear measurement operator is unknown.
The proposed method is problem-agnostic, meaning that a pre-trained diffusion model can be applied to various inverse problems without fine-tuning.
arXiv Detail & Related papers (2023-01-30T06:27:48Z) - Causal Graph Discovery from Self and Mutually Exciting Time Series [10.410454851418548]
We develop a non-asymptotic recovery guarantee and quantifiable uncertainty by solving a linear program.
We demonstrate the effectiveness of our approach in recovering highly interpretable causal DAGs over Sepsis Associated Derangements (SADs)
arXiv Detail & Related papers (2023-01-26T16:15:27Z) - MissDAG: Causal Discovery in the Presence of Missing Data with
Continuous Additive Noise Models [78.72682320019737]
We develop a general method, which we call MissDAG, to perform causal discovery from data with incomplete observations.
MissDAG maximizes the expected likelihood of the visible part of observations under the expectation-maximization framework.
We demonstrate the flexibility of MissDAG for incorporating various causal discovery algorithms and its efficacy through extensive simulations and real data experiments.
arXiv Detail & Related papers (2022-05-27T09:59:46Z) - BCD Nets: Scalable Variational Approaches for Bayesian Causal Discovery [97.79015388276483]
A structural equation model (SEM) is an effective framework to reason over causal relationships represented via a directed acyclic graph (DAG)
Recent advances enabled effective maximum-likelihood point estimation of DAGs from observational data.
We propose BCD Nets, a variational framework for estimating a distribution over DAGs characterizing a linear-Gaussian SEM.
arXiv Detail & Related papers (2021-12-06T03:35:21Z) - Causal Graph Discovery from Self and Mutually Exciting Time Series [12.802653884445132]
We develop a non-asymptotic recovery guarantee and quantifiable uncertainty by solving a linear program.
We demonstrate the effectiveness of our approach in recovering highly interpretable causal DAGs over Sepsis Associated Derangements (SADs)
arXiv Detail & Related papers (2021-06-04T16:59:24Z) - CASTLE: Regularization via Auxiliary Causal Graph Discovery [89.74800176981842]
We introduce Causal Structure Learning (CASTLE) regularization and propose to regularize a neural network by jointly learning the causal relationships between variables.
CASTLE efficiently reconstructs only the features in the causal DAG that have a causal neighbor, whereas reconstruction-based regularizers suboptimally reconstruct all input features.
arXiv Detail & Related papers (2020-09-28T09:49:38Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.