Shortcuts for causal discovery of nonlinear models by score matching

• URL: http://arxiv.org/abs/2310.14246v1
• Date: Sun, 22 Oct 2023 10:09:52 GMT
• Title: Shortcuts for causal discovery of nonlinear models by score matching
• Authors: Francesco Montagna, Nicoletta Noceti, Lorenzo Rosasco, Francesco Locatello
• Abstract summary: We define and characterize a score-sortability pattern of nonlinear additive noise models. We show the score-sortability of the most common synthetic benchmarks in the literature. Our findings remark the lack of diversity in the data as an important limitation in the evaluation of nonlinear causal discovery approaches.
• Score: 32.01302470630594
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The use of simulated data in the field of causal discovery is ubiquitous due to the scarcity of annotated real data. Recently, Reisach et al., 2021 highlighted the emergence of patterns in simulated linear data, which displays increasing marginal variance in the casual direction. As an ablation in their experiments, Montagna et al., 2023 found that similar patterns may emerge in nonlinear models for the variance of the score vector $\nabla \log p_{\mathbf{X}}$, and introduced the ScoreSort algorithm. In this work, we formally define and characterize this score-sortability pattern of nonlinear additive noise models. We find that it defines a class of identifiable (bivariate) causal models overlapping with nonlinear additive noise models. We theoretically demonstrate the advantages of ScoreSort in terms of statistical efficiency compared to prior state-of-the-art score matching-based methods and empirically show the score-sortability of the most common synthetic benchmarks in the literature. Our findings remark (1) the lack of diversity in the data as an important limitation in the evaluation of nonlinear causal discovery approaches, (2) the importance of thoroughly testing different settings within a problem class, and (3) the importance of analyzing statistical properties in causal discovery, where research is often limited to defining identifiability conditions of the model.

Related papers

• Score matching through the roof: linear, nonlinear, and latent variables causal discovery [18.46845413928147]
  Causal discovery from observational data holds great promise. Existing methods rely on strong assumptions about the underlying causal structure. We propose a flexible algorithm for causal discovery across linear, nonlinear, and latent variable models.
  arXiv  Detail & Related papers  (2024-07-26T14:09:06Z)
• Demystifying amortized causal discovery with transformers [21.058343547918053]
  Supervised learning approaches for causal discovery from observational data often achieve competitive performance. In this work, we investigate CSIvA, a transformer-based model promising to train on synthetic data and transfer to real data. We bridge the gap with existing identifiability theory and show that constraints on the training data distribution implicitly define a prior on the test observations.
  arXiv  Detail & Related papers  (2024-05-27T08:17:49Z)
• Sample, estimate, aggregate: A recipe for causal discovery foundation models [28.116832159265964]
  We train a supervised model that learns to predict a larger causal graph from the outputs of classical causal discovery algorithms run over subsets of variables. Our approach is enabled by the observation that typical errors in the outputs of classical methods remain comparable across datasets. Experiments on real and synthetic data demonstrate that this model maintains high accuracy in the face of misspecification or distribution shift.
  arXiv  Detail & Related papers  (2024-02-02T21:57:58Z)
• Sample Complexity Bounds for Score-Matching: Causal Discovery and Generative Modeling [82.36856860383291]
  We demonstrate that accurate estimation of the score function is achievable by training a standard deep ReLU neural network. We establish bounds on the error rate of recovering causal relationships using the score-matching-based causal discovery method.
  arXiv  Detail & Related papers  (2023-10-27T13:09:56Z)
• Assessing the overall and partial causal well-specification of nonlinear additive noise models [4.13592995550836]
  We aim to identify predictor variables for which we can infer the causal effect even in cases of such misspecifications. We propose an algorithm for finite sample data, discuss its properties, and illustrate its performance on simulated and real data.
  arXiv  Detail & Related papers  (2023-10-25T09:44:16Z)
• Learning to Bound Counterfactual Inference in Structural Causal Models from Observational and Randomised Data [64.96984404868411]
  We derive a likelihood characterisation for the overall data that leads us to extend a previous EM-based algorithm. The new algorithm learns to approximate the (unidentifiability) region of model parameters from such mixed data sources. It delivers interval approximations to counterfactual results, which collapse to points in the identifiable case.
  arXiv  Detail & Related papers  (2022-12-06T12:42:11Z)
• Causal Inference via Nonlinear Variable Decorrelation for Healthcare Applications [60.26261850082012]
  We introduce a novel method with a variable decorrelation regularizer to handle both linear and nonlinear confounding. We employ association rules as new representations using association rule mining based on the original features to increase model interpretability.
  arXiv  Detail & Related papers  (2022-09-29T17:44:14Z)
• Bivariate Causal Discovery for Categorical Data via Classification with Optimal Label Permutation [2.0305676256390934]
  We propose a novel causal model for categorical data based on a new classification model, termed classification with optimal label permutation (COLP) A simple learning algorithm via comparing likelihood functions of causal and anti-causal models suffices to learn the causal direction.
  arXiv  Detail & Related papers  (2022-09-18T15:04:55Z)
• Amortized Inference for Causal Structure Learning [72.84105256353801]
  Learning causal structure poses a search problem that typically involves evaluating structures using a score or independence test. We train a variational inference model to predict the causal structure from observational/interventional data. Our models exhibit robust generalization capabilities under substantial distribution shift.
  arXiv  Detail & Related papers  (2022-05-25T17:37:08Z)
• Estimation of Bivariate Structural Causal Models by Variational Gaussian Process Regression Under Likelihoods Parametrised by Normalising Flows [74.85071867225533]
  Causal mechanisms can be described by structural causal models. One major drawback of state-of-the-art artificial intelligence is its lack of explainability.
  arXiv  Detail & Related papers  (2021-09-06T14:52:58Z)
• Good Classifiers are Abundant in the Interpolating Regime [64.72044662855612]
  We develop a methodology to compute precisely the full distribution of test errors among interpolating classifiers. We find that test errors tend to concentrate around a small typical value $varepsilon*$, which deviates substantially from the test error of worst-case interpolating model. Our results show that the usual style of analysis in statistical learning theory may not be fine-grained enough to capture the good generalization performance observed in practice.
  arXiv  Detail & Related papers  (2020-06-22T21:12:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.