Counterfactual Causal Inference in Natural Language with Large Language Models

• URL: http://arxiv.org/abs/2410.06392v1
• Date: Tue, 8 Oct 2024 21:53:07 GMT
• Title: Counterfactual Causal Inference in Natural Language with Large Language Models
• Authors: Gaël Gendron, Jože M. Rožanec, Michael Witbrock, Gillian Dobbie,
• Abstract summary: We propose an end-to-end causal structure discovery and causal inference method from natural language. We first use an LLM to extract the instantiated causal variables from text data and build a causal graph. We then conduct counterfactual inference on the estimated graph.
• Score: 9.153187514369849
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Causal structure discovery methods are commonly applied to structured data where the causal variables are known and where statistical testing can be used to assess the causal relationships. By contrast, recovering a causal structure from unstructured natural language data such as news articles contains numerous challenges due to the absence of known variables or counterfactual data to estimate the causal links. Large Language Models (LLMs) have shown promising results in this direction but also exhibit limitations. This work investigates LLM's abilities to build causal graphs from text documents and perform counterfactual causal inference. We propose an end-to-end causal structure discovery and causal inference method from natural language: we first use an LLM to extract the instantiated causal variables from text data and build a causal graph. We merge causal graphs from multiple data sources to represent the most exhaustive set of causes possible. We then conduct counterfactual inference on the estimated graph. The causal graph conditioning allows reduction of LLM biases and better represents the causal estimands. We use our method to show that the limitations of LLMs in counterfactual causal reasoning come from prediction errors and propose directions to mitigate them. We demonstrate the applicability of our method on real-world news articles.

Related papers

• Causal Agent based on Large Language Model [30.81702479532088]
  Large language models (LLMs) have achieved significant success across various domains. The inherent complexity of causal problems and causal theory poses challenges in accurately describing them in natural language. We have equipped the LLM with causal tools within an agent framework, named the Causal Agent, enabling it to tackle causal problems.
  arXiv  Detail & Related papers  (2024-08-13T12:22:26Z)
• Large Language Models for Constrained-Based Causal Discovery [4.858756226945995]
  Causality is essential for understanding complex systems, such as the economy, the brain, and the climate. This work explores the capabilities of Large Language Models (LLMs) as an alternative to domain experts for causal graph generation.
  arXiv  Detail & Related papers  (2024-06-11T15:45:24Z)
• ALCM: Autonomous LLM-Augmented Causal Discovery Framework [2.1470800327528843]
  We introduce a new framework, named Autonomous LLM-Augmented Causal Discovery Framework (ALCM), to synergize data-driven causal discovery algorithms and Large Language Models. The ALCM consists of three integral components: causal structure learning, causal wrapper, and LLM-driven causal refiner. We evaluate the ALCM framework by implementing two demonstrations on seven well-known datasets.
  arXiv  Detail & Related papers  (2024-05-02T21:27:45Z)
• Zero-shot Causal Graph Extrapolation from Text via LLMs [50.596179963913045]
  We evaluate the ability of large language models (LLMs) to infer causal relations from natural language. LLMs show competitive performance in a benchmark of pairwise relations without needing (explicit) training samples. We extend our approach to extrapolating causal graphs through iterated pairwise queries.
  arXiv  Detail & Related papers  (2023-12-22T13:14:38Z)
• CLadder: Assessing Causal Reasoning in Language Models [82.8719238178569]
  We investigate whether large language models (LLMs) can coherently reason about causality. We propose a new NLP task, causal inference in natural language, inspired by the "causal inference engine" postulated by Judea Pearl et al.
  arXiv  Detail & Related papers  (2023-12-07T15:12:12Z)
• Can We Utilize Pre-trained Language Models within Causal Discovery Algorithms? [0.2303687191203919]
  Causal reasoning of Pre-trained Language Models (PLMs) relies solely on text-based descriptions. We propose a new framework that integrates prior knowledge obtained from PLM with a causal discovery algorithm.
  arXiv  Detail & Related papers  (2023-11-19T03:31:30Z)
• Inducing Causal Structure for Abstractive Text Summarization [76.1000380429553]
  We introduce a Structural Causal Model (SCM) to induce the underlying causal structure of the summarization data. We propose a Causality Inspired Sequence-to-Sequence model (CI-Seq2Seq) to learn the causal representations that can mimic the causal factors. Experimental results on two widely used text summarization datasets demonstrate the advantages of our approach.
  arXiv  Detail & Related papers  (2023-08-24T16:06:36Z)
• Can Large Language Models Infer Causation from Correlation? [104.96351414570239]
  We test the pure causal inference skills of large language models (LLMs) We formulate a novel task Corr2Cause, which takes a set of correlational statements and determines the causal relationship between the variables. We show that these models achieve almost close to random performance on the task.
  arXiv  Detail & Related papers  (2023-06-09T12:09:15Z)
• Systematic Evaluation of Causal Discovery in Visual Model Based Reinforcement Learning [76.00395335702572]
  A central goal for AI and causality is the joint discovery of abstract representations and causal structure. Existing environments for studying causal induction are poorly suited for this objective because they have complicated task-specific causal graphs. In this work, our goal is to facilitate research in learning representations of high-level variables as well as causal structures among them.
  arXiv  Detail & Related papers  (2021-07-02T05:44:56Z)
• Causal Expectation-Maximisation [70.45873402967297]
  We show that causal inference is NP-hard even in models characterised by polytree-shaped graphs. We introduce the causal EM algorithm to reconstruct the uncertainty about the latent variables from data about categorical manifest variables. We argue that there appears to be an unnoticed limitation to the trending idea that counterfactual bounds can often be computed without knowledge of the structural equations.
  arXiv  Detail & Related papers  (2020-11-04T10:25:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.