Time Series Causal Discovery via Context-Conditioned and Causality-Augmented Pretraining

• URL: http://arxiv.org/abs/2605.26759v1
• Date: Tue, 26 May 2026 09:30:42 GMT
• Title: Time Series Causal Discovery via Context-Conditioned and Causality-Augmented Pretraining
• Abstract summary: Causal discovery from time series is critical for many real-world applications, such as tracing the root causes of anomalies.<n>We propose textbfPTCD, a novel textbfPretraining framework for textbfTime-series textbfCausal textbfDiscovery.
• Score: 17.01926647313838
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Causal discovery from time series is critical for many real-world applications, such as tracing the root causes of anomalies. Existing approaches typically rely on dataset-specific optimization, making it difficult to transfer their causal discovery capabilities to new time series governed by diverse causal mechanisms. In this paper, we propose \textbf{PTCD}, a novel \textbf{P}retraining framework for \textbf{T}ime-series \textbf{C}ausal \textbf{D}iscovery, which improves cross-task generalization through context-conditioned modeling and transferable causal augmentation. To model complex temporal causal dependencies, PTCD employs a dual-scale iterative attention mechanism to capture window-level causal relationships, and a Gaussian mixture with a context-level routing mechanism to handle heterogeneous exogenous distributions. To further address distribution shifts across causal graphs, PTCD adopts a pretraining paradigm on synthetic datasets that integrates intervention-based learning and a causal mixup strategy, promoting stable causal discovery and stronger generalization. Extensive experiments on multiple real-world out-of-distribution (OOD) datasets demonstrate that PTCD excels in both causal discovery and root cause identification.

Related papers

• Causal Discovery with Inverted Self-attention for Multivariate Time Series [21.018583540683917]
  We propose a novel framework that leverages self-attention mechanisms within the transformer architecture for causal discovery.<n>Our approach emphasizes latent and indirect causal relationships by inverting tokens and inducing sparsity in attention scores.<n>We also develop a global causal algorithm to identify global causal links, providing a holistic metric for causal influence.
  arXiv  Detail & Related papers  (2026-07-30T13:49:46Z)
• CDFM: Towards a General-Purpose Causal Discovery Foundation Model [62.32138962508216]
  Causal discovery is the process of recovering underlying causal structures from observational data.<n>We formulate the Causal Discovery Foundation Model (CDFM) as a unified, general-purpose framework for zero-shot structural inference.<n>CDFM consistently outperforms traditional algorithms, driving a paradigm shift toward a general-purpose causal discovery foundation model.
  arXiv  Detail & Related papers  (2026-07-13T12:58:28Z)
• CausalMoE: A Billion-Scale Multimodal Foundation Model for Granger Causal Discovery with Pattern-Routed Heterogeneous Experts [25.563603697914242]
  We propose CausalMoE, a billion-scale multimodal Granger causal foundation model.<n>CaulMoE introduces a Pattern-Routed Mixture of Heterogeneous Experts, which dynamically identifies latent temporal patterns.<n>We also design a Causality-Aware Self-Attention mechanism operating across variables, yielding sparse Granger causal graphs.
  arXiv  Detail & Related papers  (2026-06-11T07:57:23Z)
• Causal Ensemble Agent: Hierarchical Causal Discovery with LLM-guided Expert Reweighting [95.24017293537025]
  Causal Ensemble Agent (CEA) is a novel framework that aggregates structural insights from statistical discovery experts across different graph levels.<n>CEA achieves the strongest overall performance across a wide range of causal discovery methods.
  arXiv  Detail & Related papers  (2026-06-09T09:09:07Z)
• TabCausal: Pretraining Across Causal Environments for Tabular Causal Discovery [47.81267771630766]
  Causal discovery foundation models (CDFMs) seek to amortize this problem by mapping a dataset directly to a causal graph in a single forward pass.<n>We propose TabCausal, a data-driven CDFM trained with broad causal pretraining over diverse graph priors.
  arXiv  Detail & Related papers  (2026-05-29T11:05:43Z)
• TTCD:Transformer Integrated Temporal Causal Discovery from Non-Stationary Time Series Data [0.18097009159779617]
  We propose an end-to-end approach that learns contemporaneous and lagged causal relations from non-stationary time series.<n>A key innovation is causal signal distillation, to distill essential causal signals through the reconstruction process of the transformer decoder.<n>Experiments on synthetic, benchmark, and real world datasets show that TTCD consistently outperforms state-of-the-art baselines in both accuracy and consistency with domain knowledge.
  arXiv  Detail & Related papers  (2026-04-27T19:44:38Z)
• Spatio-Temporal Hierarchical Causal Models [12.518908171952113]
  In this paper, we introduce Spatio-Temporal Hierarchical Causal Models (ST-HC-Ms), a novel graphical framework that extends hierarchical causal modeling to thetemporal domain.<n>At the core of our approach is the Spatio-Temporal Collapse Theorem, which shows that a complex ST-HC-Ms converges to a simpler flat causal model as the amount data increases.<n>We validate our framework on both synthetic and real-world datasets, demonstrating its potential for robust causal inference in complex dynamic systems.
  arXiv  Detail & Related papers  (2025-11-25T17:56:43Z)
• Learning Time-Aware Causal Representation for Model Generalization in Evolving Domains [50.66049136093248]
  We develop a time-aware structural causal model (SCM) that incorporates dynamic causal factors and the causal mechanism drifts.<n>We show that our method can yield the optimal causal predictor for each time domain.<n>Results on both synthetic and real-world datasets exhibit that SYNC can achieve superior temporal generalization performance.
  arXiv  Detail & Related papers  (2025-06-21T14:05:37Z)
• Multi-modal Causal Structure Learning and Root Cause Analysis [67.67578590390907]
  We propose Mulan, a unified multi-modal causal structure learning method for root cause localization. We leverage a log-tailored language model to facilitate log representation learning, converting log sequences into time-series data. We also introduce a novel key performance indicator-aware attention mechanism for assessing modality reliability and co-learning a final causal graph.
  arXiv  Detail & Related papers  (2024-02-04T05:50:38Z)
• Causal Temporal Regime Structure Learning [49.77103348208835]
  We present CASTOR, a novel method that concurrently learns the Directed Acyclic Graph (DAG) for each regime.<n>We establish the identifiability of the regimes and DAGs within our framework.<n>Experiments show that CASTOR consistently outperforms existing causal discovery models.
  arXiv  Detail & Related papers  (2023-11-02T17:26:49Z)
• Causality-Based Multivariate Time Series Anomaly Detection [63.799474860969156]
  We formulate the anomaly detection problem from a causal perspective and view anomalies as instances that do not follow the regular causal mechanism to generate the multivariate data. We then propose a causality-based anomaly detection approach, which first learns the causal structure from data and then infers whether an instance is an anomaly relative to the local causal mechanism. We evaluate our approach with both simulated and public datasets as well as a case study on real-world AIOps applications.
  arXiv  Detail & Related papers  (2022-06-30T06:00:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.