Embracing the black box: Heading towards foundation models for causal discovery from time series data

• URL: http://arxiv.org/abs/2402.09305v1
• Date: Wed, 14 Feb 2024 16:49:13 GMT
• Title: Embracing the black box: Heading towards foundation models for causal discovery from time series data
• Authors: Gideon Stein, Maha Shadaydeh, Joachim Denzler
• Abstract summary: Causal Pretraining is a methodology that aims to learn a direct mapping from time series to the underlying causal graphs in a supervised manner. Our empirical findings suggest that causal discovery in a supervised manner is possible, assuming that the training and test time series samples share most of their dynamics. We provide examples where causal discovery for real-world data with causally pretrained neural networks is possible within limits.
• Score: 8.073449277052495
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Causal discovery from time series data encompasses many existing solutions, including those based on deep learning techniques. However, these methods typically do not endorse one of the most prevalent paradigms in deep learning: End-to-end learning. To address this gap, we explore what we call Causal Pretraining. A methodology that aims to learn a direct mapping from multivariate time series to the underlying causal graphs in a supervised manner. Our empirical findings suggest that causal discovery in a supervised manner is possible, assuming that the training and test time series samples share most of their dynamics. More importantly, we found evidence that the performance of Causal Pretraining can increase with data and model size, even if the additional data do not share the same dynamics. Further, we provide examples where causal discovery for real-world data with causally pretrained neural networks is possible within limits. We argue that this hints at the possibility of a foundation model for causal discovery.

Related papers

• StiefelGen: A Simple, Model Agnostic Approach for Time Series Data Augmentation over Riemannian Manifolds [10.751130375035558]
  Three common approaches for time series data augmentation include: (i) Constructing a physics-based model and then imbuing uncertainty over the coefficient space (for example), (ii) Adding noise to the observed data set(s), and (iii) Having access to ample amounts of time series data sets from which a robust generative neural network model can be trained. We propose a methodology which attempts to simultaneously tackle all three of these previous limitations to a large extent. The method relies upon the well-studied matrix differential geometry of the Stiefel manifold, as it proposes a simple way in which time series signals can placed
  arXiv  Detail & Related papers  (2024-02-29T15:52:21Z)
• Multi-modal Causal Structure Learning and Root Cause Analysis [67.67578590390907]
  We propose Mulan, a unified multi-modal causal structure learning method for root cause localization. We leverage a log-tailored language model to facilitate log representation learning, converting log sequences into time-series data. We also introduce a novel key performance indicator-aware attention mechanism for assessing modality reliability and co-learning a final causal graph.
  arXiv  Detail & Related papers  (2024-02-04T05:50:38Z)
• Sample, estimate, aggregate: A recipe for causal discovery foundation models [28.116832159265964]
  We train a supervised model that learns to predict a larger causal graph from the outputs of classical causal discovery algorithms run over subsets of variables. Our approach is enabled by the observation that typical errors in the outputs of classical methods remain comparable across datasets. Experiments on real and synthetic data demonstrate that this model maintains high accuracy in the face of misspecification or distribution shift.
  arXiv  Detail & Related papers  (2024-02-02T21:57:58Z)
• Discovering Mixtures of Structural Causal Models from Time Series Data [23.18511951330646]
  We propose a general variational inference-based framework called MCD to infer the underlying causal models. Our approach employs an end-to-end training process that maximizes an evidence-lower bound for the data likelihood. We demonstrate that our method surpasses state-of-the-art benchmarks in causal discovery tasks.
  arXiv  Detail & Related papers  (2023-10-10T05:13:10Z)
• Deep networks for system identification: a Survey [56.34005280792013]
  System identification learns mathematical descriptions of dynamic systems from input-output data. Main aim of the identified model is to predict new data from previous observations. We discuss architectures commonly adopted in the literature, like feedforward, convolutional, and recurrent networks.
  arXiv  Detail & Related papers  (2023-01-30T12:38:31Z)
• Principled Knowledge Extrapolation with GANs [92.62635018136476]
  We study counterfactual synthesis from a new perspective of knowledge extrapolation. We show that an adversarial game with a closed-form discriminator can be used to address the knowledge extrapolation problem. Our method enjoys both elegant theoretical guarantees and superior performance in many scenarios.
  arXiv  Detail & Related papers  (2022-05-21T08:39:42Z)
• Neural Additive Vector Autoregression Models for Causal Discovery in Time Series [1.160208922584163]
  We propose a neural approach to causal structure learning that can discover nonlinear relationships. We train deep neural networks that extract the (additive) Granger causal influences from the time evolution in time series. The method achieves state-of-the-art results on various benchmark data sets for causal discovery.
  arXiv  Detail & Related papers  (2020-10-19T12:44:25Z)
• Amortized Causal Discovery: Learning to Infer Causal Graphs from Time-Series Data [63.15776078733762]
  We propose Amortized Causal Discovery, a novel framework to learn to infer causal relations from time-series data. We demonstrate experimentally that this approach, implemented as a variational model, leads to significant improvements in causal discovery performance.
  arXiv  Detail & Related papers  (2020-06-18T19:59:12Z)
• Learning Causal Models Online [103.87959747047158]
  Predictive models can rely on spurious correlations in the data for making predictions. One solution for achieving strong generalization is to incorporate causal structures in the models. We propose an online algorithm that continually detects and removes spurious features.
  arXiv  Detail & Related papers  (2020-06-12T20:49:20Z)
• A Multi-Channel Neural Graphical Event Model with Negative Evidence [76.51278722190607]
  Event datasets are sequences of events of various types occurring irregularly over the time-line. We propose a non-parametric deep neural network approach in order to estimate the underlying intensity functions.
  arXiv  Detail & Related papers  (2020-02-21T23:10:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.