CausalFormer: An Interpretable Transformer for Temporal Causal Discovery

• URL: http://arxiv.org/abs/2406.16708v1
• Date: Mon, 24 Jun 2024 15:09:29 GMT
• Title: CausalFormer: An Interpretable Transformer for Temporal Causal Discovery
• Authors: Lingbai Kong, Wengen Li, Hanchen Yang, Yichao Zhang, Jihong Guan, Shuigeng Zhou,
• Abstract summary: We propose an interpretable transformer-based causal discovery model termed CausalFormer. The causality-aware transformer learns the causal representation of time series data using a prediction task. The decomposition-based causality detector interprets the global structure of the trained causality-aware transformer.
• Score: 24.608536564444137
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Temporal causal discovery is a crucial task aimed at uncovering the causal relations within time series data. The latest temporal causal discovery methods usually train deep learning models on prediction tasks to uncover the causality between time series. They capture causal relations by analyzing the parameters of some components of the trained models, e.g., attention weights and convolution weights. However, this is an incomplete mapping process from the model parameters to the causality and fails to investigate the other components, e.g., fully connected layers and activation functions, that are also significant for causal discovery. To facilitate the utilization of the whole deep learning models in temporal causal discovery, we proposed an interpretable transformer-based causal discovery model termed CausalFormer, which consists of the causality-aware transformer and the decomposition-based causality detector. The causality-aware transformer learns the causal representation of time series data using a prediction task with the designed multi-kernel causal convolution which aggregates each input time series along the temporal dimension under the temporal priority constraint. Then, the decomposition-based causality detector interprets the global structure of the trained causality-aware transformer with the proposed regression relevance propagation to identify potential causal relations and finally construct the causal graph. Experiments on synthetic, simulated, and real datasets demonstrate the state-of-the-art performance of CausalFormer on discovering temporal causality. Our code is available at https://github.com/lingbai-kong/CausalFormer.

Related papers

• CAnDOIT: Causal Discovery with Observational and Interventional Data from Time-Series [4.008958683836471]
  CAnDOIT is a causal discovery method to reconstruct causal models using both observational and interventional data. The use of interventional data in the causal analysis is crucial for real-world applications, such as robotics. A Python implementation of CAnDOIT has also been developed and is publicly available on GitHub.
  arXiv  Detail & Related papers  (2024-10-03T13:57:08Z)
• Causal discovery for time series with constraint-based model and PMIME measure [0.0]
  We present a novel approach for discovering causality in time series data that combines a causal discovery algorithm with an information theoretic-based measure. We evaluate the performance of our approach on several simulated data sets, showing promising results.
  arXiv  Detail & Related papers  (2023-05-31T09:38:50Z)
• TC-GAT: Graph Attention Network for Temporal Causality Discovery [6.974417592057705]
  Causality is frequently intertwined with temporal elements, as the progression from cause to effect is not instantaneous but rather ensconced in a temporal dimension. We propose a method for extracting causality from the text that integrates both temporal and causal relations. We present a novel model, TC-GAT, which employs a graph attention mechanism to assign weights to the temporal relationships and leverages a causal knowledge graph to determine the adjacency matrix.
  arXiv  Detail & Related papers  (2023-04-21T02:26:42Z)
• DOMINO: Visual Causal Reasoning with Time-Dependent Phenomena [59.291745595756346]
  We propose a set of visual analytics methods that allow humans to participate in the discovery of causal relations associated with windows of time delay. Specifically, we leverage a well-established method, logic-based causality, to enable analysts to test the significance of potential causes. Since an effect can be a cause of other effects, we allow users to aggregate different temporal cause-effect relations found with our method into a visual flow diagram.
  arXiv  Detail & Related papers  (2023-03-12T03:40:21Z)
• Causality-Based Multivariate Time Series Anomaly Detection [63.799474860969156]
  We formulate the anomaly detection problem from a causal perspective and view anomalies as instances that do not follow the regular causal mechanism to generate the multivariate data. We then propose a causality-based anomaly detection approach, which first learns the causal structure from data and then infers whether an instance is an anomaly relative to the local causal mechanism. We evaluate our approach with both simulated and public datasets as well as a case study on real-world AIOps applications.
  arXiv  Detail & Related papers  (2022-06-30T06:00:13Z)
• Amortized Inference for Causal Structure Learning [72.84105256353801]
  Learning causal structure poses a search problem that typically involves evaluating structures using a score or independence test. We train a variational inference model to predict the causal structure from observational/interventional data. Our models exhibit robust generalization capabilities under substantial distribution shift.
  arXiv  Detail & Related papers  (2022-05-25T17:37:08Z)
• Neural Additive Vector Autoregression Models for Causal Discovery in Time Series [1.160208922584163]
  We propose a neural approach to causal structure learning that can discover nonlinear relationships. We train deep neural networks that extract the (additive) Granger causal influences from the time evolution in time series. The method achieves state-of-the-art results on various benchmark data sets for causal discovery.
  arXiv  Detail & Related papers  (2020-10-19T12:44:25Z)
• Amortized Causal Discovery: Learning to Infer Causal Graphs from Time-Series Data [63.15776078733762]
  We propose Amortized Causal Discovery, a novel framework to learn to infer causal relations from time-series data. We demonstrate experimentally that this approach, implemented as a variational model, leads to significant improvements in causal discovery performance.
  arXiv  Detail & Related papers  (2020-06-18T19:59:12Z)
• Learning Causal Models Online [103.87959747047158]
  Predictive models can rely on spurious correlations in the data for making predictions. One solution for achieving strong generalization is to incorporate causal structures in the models. We propose an online algorithm that continually detects and removes spurious features.
  arXiv  Detail & Related papers  (2020-06-12T20:49:20Z)
• A Critical View of the Structural Causal Model [89.43277111586258]
  We show that one can identify the cause and the effect without considering their interaction at all. We propose a new adversarial training method that mimics the disentangled structure of the causal model. Our multidimensional method outperforms the literature methods on both synthetic and real world datasets.
  arXiv  Detail & Related papers  (2020-02-23T22:52:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.