Differentiable Invariant Causal Discovery
- URL: http://arxiv.org/abs/2205.15638v2
- Date: Wed, 1 Jun 2022 02:19:59 GMT
- Title: Differentiable Invariant Causal Discovery
- Authors: Yu Wang, An Zhang, Xiang Wang, Xiangnan He, Tat-Seng Chua
- Abstract summary: Learning causal structure from observational data is a fundamental challenge in machine learning.
This paper proposes Differentiable Invariant Causal Discovery (DICD) to avoid learning spurious edges and wrong causal directions.
Extensive experiments on synthetic and real-world datasets verify that DICD outperforms state-of-the-art causal discovery methods up to 36% in SHD.
- Score: 106.87950048845308
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Learning causal structure from observational data is a fundamental challenge
in machine learning. The majority of commonly used differentiable causal
discovery methods are non-identifiable, turning this problem into a continuous
optimization task prone to data biases. In many real-life situations, data is
collected from different environments, in which the functional relations remain
consistent across environments, while the distribution of additive noises may
vary. This paper proposes Differentiable Invariant Causal Discovery (DICD),
utilizing the multi-environment information based on a differentiable framework
to avoid learning spurious edges and wrong causal directions. Specifically,
DICD aims to discover the environment-invariant causation while removing the
environment-dependent correlation. We further formulate the constraint that
enforces the target structure equation model to maintain optimal across the
environments. Theoretical guarantees for the identifiability of proposed DICD
are provided under mild conditions with enough environments. Extensive
experiments on synthetic and real-world datasets verify that DICD outperforms
state-of-the-art causal discovery methods up to 36% in SHD. Our code will be
open-sourced upon acceptance.
Related papers
- The Implicit Bias of Heterogeneity towards Invariance: A Study of Multi-Environment Matrix Sensing [9.551225697705199]
This paper studies the implicit bias of Gradient Descent (SGD) over heterogeneous data and shows that the implicit bias drives the model learning towards an invariant solution.
Specifically, we theoretically investigate the multi-environment low-rank matrix sensing problem where in each environment, the signal comprises (i) a lower-rank invariant part shared across all environments; and (ii) a significantly varying environment-dependent spurious component.
The key insight is, through simply employing the large step size large-batch SGD sequentially in each environment without any explicit regularization, the oscillation caused by heterogeneity can provably prevent model learning spurious signals.
arXiv Detail & Related papers (2024-03-03T07:38:24Z) - Federated Causal Discovery from Heterogeneous Data [70.31070224690399]
We propose a novel FCD method attempting to accommodate arbitrary causal models and heterogeneous data.
These approaches involve constructing summary statistics as a proxy of the raw data to protect data privacy.
We conduct extensive experiments on synthetic and real datasets to show the efficacy of our method.
arXiv Detail & Related papers (2024-02-20T18:53:53Z) - Towards stable real-world equation discovery with assessing
differentiating quality influence [52.2980614912553]
We propose alternatives to the commonly used finite differences-based method.
We evaluate these methods in terms of applicability to problems, similar to the real ones, and their ability to ensure the convergence of equation discovery algorithms.
arXiv Detail & Related papers (2023-11-09T23:32:06Z) - Structural restrictions in local causal discovery: identifying direct causes of a target variable [0.9208007322096533]
Learning a set of direct causes of a target variable from an observational joint distribution is a fundamental problem in science.
Here, we are only interested in identifying the direct causes of one target variable, not the full DAG.
This allows us to relax the identifiability assumptions and develop possibly faster and more robust algorithms.
arXiv Detail & Related papers (2023-07-29T18:31:35Z) - DCID: Deep Canonical Information Decomposition [84.59396326810085]
We consider the problem of identifying the signal shared between two one-dimensional target variables.
We propose ICM, an evaluation metric which can be used in the presence of ground-truth labels.
We also propose Deep Canonical Information Decomposition (DCID) - a simple, yet effective approach for learning the shared variables.
arXiv Detail & Related papers (2023-06-27T16:59:06Z) - Deep Generative Model for Simultaneous Range Error Mitigation and
Environment Identification [29.827191184889898]
This paper proposes a deep generative model (DGM) for simultaneous range error mitigation and environment identification.
Experiments on a general Ultra-wideband dataset demonstrate the superior performance on range error mitigation, scalability to different environments, and novel capability on simultaneous environment identification.
arXiv Detail & Related papers (2023-05-23T10:16:22Z) - Decorr: Environment Partitioning for Invariant Learning and OOD Generalization [10.799855921851332]
Invariant learning methods are aimed at identifying a consistent predictor across multiple environments.
When environments aren't inherent in the data, practitioners must define them manually.
This environment partitioning affects invariant learning's efficacy but remains underdiscussed.
In this paper, we suggest partitioning the dataset into several environments by isolating low-correlation data subsets.
arXiv Detail & Related papers (2022-11-18T06:49:35Z) - Unleashing the Power of Graph Data Augmentation on Covariate
Distribution Shift [50.98086766507025]
We propose a simple-yet-effective data augmentation strategy, Adversarial Invariant Augmentation (AIA)
AIA aims to extrapolate and generate new environments, while concurrently preserving the original stable features during the augmentation process.
arXiv Detail & Related papers (2022-11-05T07:55:55Z) - ZIN: When and How to Learn Invariance by Environment Inference? [24.191152823045385]
Invariant learning methods have proposed to learn robust and invariant models based on environment partition.
We show that learning invariant features under this circumstance is fundamentally impossible without further inductive biases or additional information.
We propose a framework to jointly learn environment partition and invariant representation, assisted by additional auxiliary information.
arXiv Detail & Related papers (2022-03-11T10:00:33Z) - Variational Causal Networks: Approximate Bayesian Inference over Causal
Structures [132.74509389517203]
We introduce a parametric variational family modelled by an autoregressive distribution over the space of discrete DAGs.
In experiments, we demonstrate that the proposed variational posterior is able to provide a good approximation of the true posterior.
arXiv Detail & Related papers (2021-06-14T17:52:49Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.