Use What You Know: Causal Foundation Models with Partial Graphs

• URL: http://arxiv.org/abs/2602.14972v1
• Date: Mon, 16 Feb 2026 17:56:37 GMT
• Title: Use What You Know: Causal Foundation Models with Partial Graphs
• Authors: Arik Reuter, Anish Dhir, Cristiana Diaconu, Jake Robertson, Ole Ossen, Frank Hutter, Adrian Weller, Mark van der Wilk, Bernhard Schölkopf,
• Abstract summary: Recently proposed Causal Foundation Models (CFMs) promise a more unified approach by amortising causal discovery and inference in a single step.<n>We bridge this gap by introducing methods to condition CFMs on causal information, such as the causal graph or more readily available ancestral information.
• Score: 97.91863420927866
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Estimating causal quantities traditionally relies on bespoke estimators tailored to specific assumptions. Recently proposed Causal Foundation Models (CFMs) promise a more unified approach by amortising causal discovery and inference in a single step. However, in their current state, they do not allow for the incorporation of any domain knowledge, which can lead to suboptimal predictions. We bridge this gap by introducing methods to condition CFMs on causal information, such as the causal graph or more readily available ancestral information. When access to complete causal graph information is too strict a requirement, our approach also effectively leverages partial causal information. We systematically evaluate conditioning strategies and find that injecting learnable biases into the attention mechanism is the most effective method to utilise full and partial causal information. Our experiments show that this conditioning allows a general-purpose CFM to match the performance of specialised models trained on specific causal structures. Overall, our approach addresses a central hurdle on the path towards all-in-one causal foundation models: the capability to answer causal queries in a data-driven manner while effectively leveraging any amount of domain expertise.

Related papers

• Leveraging Large Language Models for Causal Discovery: a Constraint-based, Argumentation-driven Approach [9.175642602891939]
  Causal Assumption-based Argumentation (ABA) is a framework that uses symbolic reasoning to ensure correspondence between input constraints and output graphs.<n>We explore the use of large language models (LLMs) as imperfect experts for Causal ABA, eliciting semantic structural priors from variable names and descriptions.<n> Experiments on standard benchmarks and semantically grounded synthetic graphs demonstrate state-of-the-art performance.
  arXiv  Detail & Related papers  (2026-02-18T14:15:21Z)
• CAT: Causal Attention Tuning For Injecting Fine-grained Causal Knowledge into Large Language Models [42.12079243701232]
  Causal Attention Tuning (CAT) is a novel approach that injects fine-grained causal knowledge into the attention mechanism.<n>We propose an automated pipeline that leverages human priors to automatically generate token-level causal signals.<n>Cat achieves an average improvement of 5.76% on the STG dataset and 1.56% on downstream tasks.
  arXiv  Detail & Related papers  (2025-09-01T15:13:15Z)
• Retrieving Classes of Causal Orders with Inconsistent Knowledge Bases [0.8192907805418583]
  Large Language Models (LLMs) have emerged as a promising alternative for extracting causal knowledge from text-based metadata.<n>LLMs tend to be unreliable and prone to hallucinations, necessitating strategies that account for their limitations.<n>We present a new method to derive a class of acyclic tournaments, which represent plausible causal orders.
  arXiv  Detail & Related papers  (2024-12-18T16:37:51Z)
• Large Language Models for Constrained-Based Causal Discovery [4.858756226945995]
  Causality is essential for understanding complex systems, such as the economy, the brain, and the climate. This work explores the capabilities of Large Language Models (LLMs) as an alternative to domain experts for causal graph generation.
  arXiv  Detail & Related papers  (2024-06-11T15:45:24Z)
• Federated Causal Discovery from Heterogeneous Data [70.31070224690399]
  We propose a novel FCD method attempting to accommodate arbitrary causal models and heterogeneous data. These approaches involve constructing summary statistics as a proxy of the raw data to protect data privacy. We conduct extensive experiments on synthetic and real datasets to show the efficacy of our method.
  arXiv  Detail & Related papers  (2024-02-20T18:53:53Z)
• Discovering and Reasoning of Causality in the Hidden World with Large Language Models [109.62442253177376]
  We develop a new framework termed Causal representatiOn AssistanT (COAT) to propose useful measured variables for causal discovery.<n>Instead of directly inferring causality with Large language models (LLMs), COAT constructs feedback from intermediate causal discovery results to LLMs to refine the proposed variables.
  arXiv  Detail & Related papers  (2024-02-06T12:18:54Z)
• Multi-modal Causal Structure Learning and Root Cause Analysis [67.67578590390907]
  We propose Mulan, a unified multi-modal causal structure learning method for root cause localization. We leverage a log-tailored language model to facilitate log representation learning, converting log sequences into time-series data. We also introduce a novel key performance indicator-aware attention mechanism for assessing modality reliability and co-learning a final causal graph.
  arXiv  Detail & Related papers  (2024-02-04T05:50:38Z)
• Advancing Counterfactual Inference through Nonlinear Quantile Regression [77.28323341329461]
  We propose a framework for efficient and effective counterfactual inference implemented with neural networks. The proposed approach enhances the capacity to generalize estimated counterfactual outcomes to unseen data. Empirical results conducted on multiple datasets offer compelling support for our theoretical assertions.
  arXiv  Detail & Related papers  (2023-06-09T08:30:51Z)
• Active Bayesian Causal Inference [72.70593653185078]
  We propose Active Bayesian Causal Inference (ABCI), a fully-Bayesian active learning framework for integrated causal discovery and reasoning. ABCI jointly infers a posterior over causal models and queries of interest. We show that our approach is more data-efficient than several baselines that only focus on learning the full causal graph.
  arXiv  Detail & Related papers  (2022-06-04T22:38:57Z)
• Principled Knowledge Extrapolation with GANs [92.62635018136476]
  We study counterfactual synthesis from a new perspective of knowledge extrapolation. We show that an adversarial game with a closed-form discriminator can be used to address the knowledge extrapolation problem. Our method enjoys both elegant theoretical guarantees and superior performance in many scenarios.
  arXiv  Detail & Related papers  (2022-05-21T08:39:42Z)
• Uncovering Main Causalities for Long-tailed Information Extraction [14.39860866665021]
  Long-tailed distributions caused by the selection bias of a dataset may lead to incorrect correlations. This motivates us to propose counterfactual IE (CFIE), a novel framework that aims to uncover the main causalities behind data.
  arXiv  Detail & Related papers  (2021-09-11T08:08:24Z)
• Incorporating Causal Graphical Prior Knowledge into Predictive Modeling via Simple Data Augmentation [92.96204497841032]
  Causal graphs (CGs) are compact representations of the knowledge of the data generating processes behind the data distributions. We propose a model-agnostic data augmentation method that allows us to exploit the prior knowledge of the conditional independence (CI) relations. We experimentally show that the proposed method is effective in improving the prediction accuracy, especially in the small-data regime.
  arXiv  Detail & Related papers  (2021-02-27T06:13:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.