LLM-initialized Differentiable Causal Discovery

• URL: http://arxiv.org/abs/2410.21141v1
• Date: Mon, 28 Oct 2024 15:43:31 GMT
• Title: LLM-initialized Differentiable Causal Discovery
• Authors: Shiv Kampani, David Hidary, Constantijn van der Poel, Martin Ganahl, Brenda Miao,
• Abstract summary: Differentiable causal discovery (DCD) methods are effective in uncovering causal relationships from observational data. However, these approaches often suffer from limited interpretability and face challenges in incorporating domain-specific prior knowledge. We propose Large Language Models (LLMs)-based causal discovery approaches that provide useful priors but struggle with formal causal reasoning.
• Score: 0.0
• License:
• Abstract: The discovery of causal relationships between random variables is an important yet challenging problem that has applications across many scientific domains. Differentiable causal discovery (DCD) methods are effective in uncovering causal relationships from observational data; however, these approaches often suffer from limited interpretability and face challenges in incorporating domain-specific prior knowledge. In contrast, Large Language Models (LLMs)-based causal discovery approaches have recently been shown capable of providing useful priors for causal discovery but struggle with formal causal reasoning. In this paper, we propose LLM-DCD, which uses an LLM to initialize the optimization of the maximum likelihood objective function of DCD approaches, thereby incorporating strong priors into the discovery method. To achieve this initialization, we design our objective function to depend on an explicitly defined adjacency matrix of the causal graph as its only variational parameter. Directly optimizing the explicitly defined adjacency matrix provides a more interpretable approach to causal discovery. Additionally, we demonstrate higher accuracy on key benchmarking datasets of our approach compared to state-of-the-art alternatives, and provide empirical evidence that the quality of the initialization directly impacts the quality of the final output of our DCD approach. LLM-DCD opens up new opportunities for traditional causal discovery methods like DCD to benefit from future improvements in the causal reasoning capabilities of LLMs.

Related papers

• Learning to Defer for Causal Discovery with Imperfect Experts [59.071731337922664]
  We propose L2D-CD, a method for gauging the correctness of expert recommendations and optimally combining them with data-driven causal discovery results. We evaluate L2D-CD on the canonical T"ubingen pairs dataset and demonstrate its superior performance compared to both the causal discovery method and the expert used in isolation.
  arXiv  Detail & Related papers  (2025-02-18T18:55:53Z)
• Discovery of Maximally Consistent Causal Orders with Large Language Models [0.8192907805418583]
  Causal discovery is essential for understanding complex systems. Traditional methods often rely on strong, untestable assumptions. We propose a novel method to derive a class of acyclic tournaments.
  arXiv  Detail & Related papers  (2024-12-18T16:37:51Z)
• Regularized Multi-LLMs Collaboration for Enhanced Score-based Causal Discovery [13.654021365091305]
  We explore the potential of using large language models (LLMs) to enhance causal discovery approaches. We propose a general framework to utilise the capacity of not only one but multiple LLMs to augment the discovery process.
  arXiv  Detail & Related papers  (2024-11-27T01:56:21Z)
• Making Large Language Models Better Planners with Reasoning-Decision Alignment [70.5381163219608]
  We motivate an end-to-end decision-making model based on multimodality-augmented LLM. We propose a reasoning-decision alignment constraint between the paired CoTs and planning results. We dub our proposed large language planners with reasoning-decision alignment as RDA-Driver.
  arXiv  Detail & Related papers  (2024-08-25T16:43:47Z)
• Large Language Models are Effective Priors for Causal Graph Discovery [6.199818486385127]
  Causal structure discovery from observations can be improved by integrating background knowledge provided by an expert to reduce the hypothesis space. Recently, Large Language Models (LLMs) have begun to be considered as sources of prior information given the low cost of querying them relative to a human expert.
  arXiv  Detail & Related papers  (2024-05-22T11:39:11Z)
• Large Language Models for Causal Discovery: Current Landscape and Future Directions [5.540272236593385]
  Causal discovery (CD) and Large Language Models (LLMs) have emerged as transformative fields in artificial intelligence. This survey examines how LLMs are transforming CD across three key dimensions: direct causal extraction from text, integration of domain knowledge into statistical methods, and refinement of causal structures.
  arXiv  Detail & Related papers  (2024-02-16T20:48:53Z)
• Discovery of the Hidden World with Large Language Models [95.58823685009727]
  This paper presents Causal representatiOn AssistanT (COAT) that introduces large language models (LLMs) to bridge the gap. LLMs are trained on massive observations of the world and have demonstrated great capability in extracting key information from unstructured data. COAT also adopts CDs to find causal relations among the identified variables as well as to provide feedback to LLMs to iteratively refine the proposed factors.
  arXiv  Detail & Related papers  (2024-02-06T12:18:54Z)
• Causal Inference Using LLM-Guided Discovery [34.040996887499425]
  We show that the topological order over graph variables (causal order) alone suffices for causal effect inference. We propose a robust technique of obtaining causal order from Large Language Models (LLMs) Our approach significantly improves causal ordering accuracy as compared to discovery algorithms.
  arXiv  Detail & Related papers  (2023-10-23T17:23:56Z)
• Differentiable Invariant Causal Discovery [106.87950048845308]
  Learning causal structure from observational data is a fundamental challenge in machine learning. This paper proposes Differentiable Invariant Causal Discovery (DICD) to avoid learning spurious edges and wrong causal directions. Extensive experiments on synthetic and real-world datasets verify that DICD outperforms state-of-the-art causal discovery methods up to 36% in SHD.
  arXiv  Detail & Related papers  (2022-05-31T09:29:07Z)
• MissDAG: Causal Discovery in the Presence of Missing Data with Continuous Additive Noise Models [78.72682320019737]
  We develop a general method, which we call MissDAG, to perform causal discovery from data with incomplete observations. MissDAG maximizes the expected likelihood of the visible part of observations under the expectation-maximization framework. We demonstrate the flexibility of MissDAG for incorporating various causal discovery algorithms and its efficacy through extensive simulations and real data experiments.
  arXiv  Detail & Related papers  (2022-05-27T09:59:46Z)
• False Correlation Reduction for Offline Reinforcement Learning [115.11954432080749]
  We propose falSe COrrelation REduction (SCORE) for offline RL, a practically effective and theoretically provable algorithm. We empirically show that SCORE achieves the SoTA performance with 3.1x acceleration on various tasks in a standard benchmark (D4RL)
  arXiv  Detail & Related papers  (2021-10-24T15:34:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.