Causal Inference Using LLM-Guided Discovery

• URL: http://arxiv.org/abs/2310.15117v1
• Date: Mon, 23 Oct 2023 17:23:56 GMT
• Title: Causal Inference Using LLM-Guided Discovery
• Authors: Aniket Vashishtha, Abbavaram Gowtham Reddy, Abhinav Kumar, Saketh Bachu, Vineeth N Balasubramanian, Amit Sharma
• Abstract summary: We show that the topological order over graph variables (causal order) alone suffices for causal effect inference. We propose a robust technique of obtaining causal order from Large Language Models (LLMs) Our approach significantly improves causal ordering accuracy as compared to discovery algorithms.
• Score: 34.040996887499425
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: At the core of causal inference lies the challenge of determining reliable causal graphs solely based on observational data. Since the well-known backdoor criterion depends on the graph, any errors in the graph can propagate downstream to effect inference. In this work, we initially show that complete graph information is not necessary for causal effect inference; the topological order over graph variables (causal order) alone suffices. Further, given a node pair, causal order is easier to elicit from domain experts compared to graph edges since determining the existence of an edge can depend extensively on other variables. Interestingly, we find that the same principle holds for Large Language Models (LLMs) such as GPT-3.5-turbo and GPT-4, motivating an automated method to obtain causal order (and hence causal effect) with LLMs acting as virtual domain experts. To this end, we employ different prompting strategies and contextual cues to propose a robust technique of obtaining causal order from LLMs. Acknowledging LLMs' limitations, we also study possible techniques to integrate LLMs with established causal discovery algorithms, including constraint-based and score-based methods, to enhance their performance. Extensive experiments demonstrate that our approach significantly improves causal ordering accuracy as compared to discovery algorithms, highlighting the potential of LLMs to enhance causal inference across diverse fields.

Related papers

• Learning to Defer for Causal Discovery with Imperfect Experts [59.071731337922664]
  We propose L2D-CD, a method for gauging the correctness of expert recommendations and optimally combining them with data-driven causal discovery results. We evaluate L2D-CD on the canonical T"ubingen pairs dataset and demonstrate its superior performance compared to both the causal discovery method and the expert used in isolation.
  arXiv  Detail & Related papers  (2025-02-18T18:55:53Z)
• CausalGraph2LLM: Evaluating LLMs for Causal Queries [49.337170619608145]
  Causality is essential in scientific research, enabling researchers to interpret true relationships between variables. With the recent advancements in Large Language Models (LLMs), there is an increasing interest in exploring their capabilities in causal reasoning.
  arXiv  Detail & Related papers  (2024-10-21T12:12:21Z)
• Subtle Errors Matter: Preference Learning via Error-injected Self-editing [59.405145971637204]
  We propose a novel preference learning framework called eRror-Injected Self-Editing (RISE) RISE injects predefined subtle errors into pivotal tokens in reasoning or steps to construct hard pairs for error mitigation. Experiments validate the effectiveness of RISE, with preference learning on Qwen2-7B-Instruct yielding notable improvements of 3.0% on GSM8K and 7.9% on MATH with only 4.5K training samples.
  arXiv  Detail & Related papers  (2024-10-09T07:43:38Z)
• Improving LLM Reasoning through Scaling Inference Computation with Collaborative Verification [52.095460362197336]
  Large language models (LLMs) struggle with consistent and accurate reasoning. LLMs are trained primarily on correct solutions, reducing their ability to detect and learn from errors. We propose a novel collaborative method integrating Chain-of-Thought (CoT) and Program-of-Thought (PoT) solutions for verification.
  arXiv  Detail & Related papers  (2024-10-05T05:21:48Z)
• Few-shot Knowledge Graph Relational Reasoning via Subgraph Adaptation [51.47994645529258]
  Few-shot Knowledge Graph (KG) Reasoning aims to predict unseen triplets (i.e., query triplets) for rare relations in KGs. We propose SAFER (Subgraph Adaptation for Few-shot Reasoning), a novel approach that effectively adapts the information in contextualized graphs to various subgraphs.
  arXiv  Detail & Related papers  (2024-06-19T21:40:35Z)
• Large Language Models for Constrained-Based Causal Discovery [4.858756226945995]
  Causality is essential for understanding complex systems, such as the economy, the brain, and the climate. This work explores the capabilities of Large Language Models (LLMs) as an alternative to domain experts for causal graph generation.
  arXiv  Detail & Related papers  (2024-06-11T15:45:24Z)
• Large Language Models are Effective Priors for Causal Graph Discovery [6.199818486385127]
  Causal structure discovery from observations can be improved by integrating background knowledge provided by an expert to reduce the hypothesis space. Recently, Large Language Models (LLMs) have begun to be considered as sources of prior information given the low cost of querying them relative to a human expert.
  arXiv  Detail & Related papers  (2024-05-22T11:39:11Z)
• ALCM: Autonomous LLM-Augmented Causal Discovery Framework [2.1470800327528843]
  We introduce a new framework, named Autonomous LLM-Augmented Causal Discovery Framework (ALCM), to synergize data-driven causal discovery algorithms and Large Language Models. The ALCM consists of three integral components: causal structure learning, causal wrapper, and LLM-driven causal refiner. We evaluate the ALCM framework by implementing two demonstrations on seven well-known datasets.
  arXiv  Detail & Related papers  (2024-05-02T21:27:45Z)
• Achieving >97% on GSM8K: Deeply Understanding the Problems Makes LLMs Better Solvers for Math Word Problems [50.76385564061713]
  Chain-of-Thought (CoT) prompting has enhanced the performance of Large Language Models (LLMs) across various reasoning tasks. CoT usually suffers from three pitfalls: semantic misunderstanding errors, calculation errors, and step-missing errors. We propose Deeply Understanding the Problems (DUP) to improve the LLMs' math problem-solving ability by addressing semantic misunderstanding errors.
  arXiv  Detail & Related papers  (2024-04-23T12:16:05Z)
• Redefining the Shortest Path Problem Formulation of the Linear Non-Gaussian Acyclic Model: Pairwise Likelihood Ratios, Prior Knowledge, and Path Enumeration [0.0]
  The paper proposes a threefold enhancement to the LiNGAM-SPP framework. The need for parameter tuning is eliminated by using the pairwise likelihood ratio in lieu of kNN-based mutual information. The incorporation of prior knowledge is then enabled by a node-skipping strategy implemented on the graph representation of all causal orderings.
  arXiv  Detail & Related papers  (2024-04-18T05:59:28Z)
• Zero-shot Causal Graph Extrapolation from Text via LLMs [50.596179963913045]
  We evaluate the ability of large language models (LLMs) to infer causal relations from natural language. LLMs show competitive performance in a benchmark of pairwise relations without needing (explicit) training samples. We extend our approach to extrapolating causal graphs through iterated pairwise queries.
  arXiv  Detail & Related papers  (2023-12-22T13:14:38Z)
• Graph Elicitation for Guiding Multi-Step Reasoning in Large Language Models [16.432208223793666]
  Chain-of-Thought prompting along with sub-question generation and answering has enhanced multi-step reasoning capabilities. We propose a GE-Reasoning method, which directs Large Language Models to generate proper sub-questions and corresponding answers. Our approach outperforms previous CoT prompting methods and their variants on multi-hop question answering benchmark datasets.
  arXiv  Detail & Related papers  (2023-11-16T10:36:08Z)
• To Copy Rather Than Memorize: A Vertical Learning Paradigm for Knowledge Graph Completion [35.05965140700747]
  We extend embedding models by allowing to explicitly copy target information from related factual triples for more accurate prediction. We also propose a novel relative distance based negative sampling technique (ReD) for more effective optimization.
  arXiv  Detail & Related papers  (2023-05-23T14:53:20Z)
• Causal Reasoning and Large Language Models: Opening a New Frontier for Causality [29.433401785920065]
  Large language models (LLMs) can generate causal arguments with high probability. LLMs may be used by human domain experts to save effort in setting up a causal analysis.
  arXiv  Detail & Related papers  (2023-04-28T19:00:43Z)
• Query-based Instance Discrimination Network for Relational Triple Extraction [39.35417927570248]
  Joint entity and relation extraction has been a core task in the field of information extraction. Recent approaches usually consider the extraction of relational triples from a stereoscopic perspective. We propose a novel query-based approach to construct instance-level representations for relational triples.
  arXiv  Detail & Related papers  (2022-11-03T13:34:56Z)
• Walk-and-Relate: A Random-Walk-based Algorithm for Representation Learning on Sparse Knowledge Graphs [5.444459446244819]
  We propose an efficient method to augment the number of triplets to address the problem of data sparsity. We also provide approaches to accurately and efficiently filter out informative metapaths from the possible set of metapaths. The proposed approaches are model-agnostic, and the augmented training dataset can be used with any KG embedding approach out of the box.
  arXiv  Detail & Related papers  (2022-09-19T05:35:23Z)
• Multi-task Learning of Order-Consistent Causal Graphs [59.9575145128345]
  We consider the problem of discovering $K related Gaussian acyclic graphs (DAGs) Under multi-task learning setting, we propose a $l_1/l$-regularized maximum likelihood estimator (MLE) for learning $K$ linear structural equation models. We theoretically show that the joint estimator, by leveraging data across related tasks, can achieve a better sample complexity for recovering the causal order.
  arXiv  Detail & Related papers  (2021-11-03T22:10:18Z)
• Variational Causal Networks: Approximate Bayesian Inference over Causal Structures [132.74509389517203]
  We introduce a parametric variational family modelled by an autoregressive distribution over the space of discrete DAGs. In experiments, we demonstrate that the proposed variational posterior is able to provide a good approximation of the true posterior.
  arXiv  Detail & Related papers  (2021-06-14T17:52:49Z)
• Counterfactual Maximum Likelihood Estimation for Training Deep Networks [83.44219640437657]
  Deep learning models are prone to learning spurious correlations that should not be learned as predictive clues. We propose a causality-based training framework to reduce the spurious correlations caused by observable confounders. We conduct experiments on two real-world tasks: Natural Language Inference (NLI) and Image Captioning.
  arXiv  Detail & Related papers  (2021-06-07T17:47:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.