Regularized Multi-LLMs Collaboration for Enhanced Score-based Causal Discovery

• URL: http://arxiv.org/abs/2411.17989v1
• Date: Wed, 27 Nov 2024 01:56:21 GMT
• Title: Regularized Multi-LLMs Collaboration for Enhanced Score-based Causal Discovery
• Authors: Xiaoxuan Li, Yao Liu, Ruoyu Wang, Lina Yao,
• Abstract summary: We explore the potential of using large language models (LLMs) to enhance causal discovery approaches. We propose a general framework to utilise the capacity of not only one but multiple LLMs to augment the discovery process.
• Score: 13.654021365091305
• License:
• Abstract: As the significance of understanding the cause-and-effect relationships among variables increases in the development of modern systems and algorithms, learning causality from observational data has become a preferred and efficient approach over conducting randomized control trials. However, purely observational data could be insufficient to reconstruct the true causal graph. Consequently, many researchers tried to utilise some form of prior knowledge to improve causal discovery process. In this context, the impressive capabilities of large language models (LLMs) have emerged as a promising alternative to the costly acquisition of prior expert knowledge. In this work, we further explore the potential of using LLMs to enhance causal discovery approaches, particularly focusing on score-based methods, and we propose a general framework to utilise the capacity of not only one but multiple LLMs to augment the discovery process.

Related papers

• Exploring Multi-Modal Integration with Tool-Augmented LLM Agents for Precise Causal Discovery [45.777770849667775]
  Causal inference is an imperative foundation for decision-making across domains, such as smart health, AI for drug discovery and AIOps. We introduce MATMCD, a multi-agent system powered by tool-augmented LLMs. Our empirical study suggests the significant potential of multi-modality enhanced causal discovery.
  arXiv  Detail & Related papers  (2024-12-18T09:50:00Z)
• LLM-initialized Differentiable Causal Discovery [0.0]
  Differentiable causal discovery (DCD) methods are effective in uncovering causal relationships from observational data. However, these approaches often suffer from limited interpretability and face challenges in incorporating domain-specific prior knowledge. We propose Large Language Models (LLMs)-based causal discovery approaches that provide useful priors but struggle with formal causal reasoning.
  arXiv  Detail & Related papers  (2024-10-28T15:43:31Z)
• Efficient Reinforcement Learning with Large Language Model Priors [18.72288751305885]
  Large language models (LLMs) have recently emerged as powerful general-purpose tools. We propose treating LLMs as prior action distributions and integrating them into RL frameworks. We show that incorporating LLM-based action priors significantly reduces exploration and complexity optimization.
  arXiv  Detail & Related papers  (2024-10-10T13:54:11Z)
• FactorLLM: Factorizing Knowledge via Mixture of Experts for Large Language Models [50.331708897857574]
  We introduce FactorLLM, a novel approach that decomposes well-trained dense FFNs into sparse sub-networks without requiring any further modifications. FactorLLM achieves comparable performance to the source model securing up to 85% model performance while obtaining over a 30% increase in inference speed.
  arXiv  Detail & Related papers  (2024-08-15T16:45:16Z)
• Knowledge Graph Structure as Prompt: Improving Small Language Models Capabilities for Knowledge-based Causal Discovery [10.573861741540853]
  KG Structure as Prompt is a novel approach for integrating structural information from a knowledge graph, such as common neighbor nodes and metapaths, into prompt-based learning. Experimental results on three types of biomedical and open-domain datasets under few-shot settings demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2024-07-26T14:07:00Z)
• Causal Graph Discovery with Retrieval-Augmented Generation based Large Language Models [23.438388321411693]
  Causal graph recovery is traditionally done using statistical estimation-based methods or based on individual's knowledge about variables of interests. We propose a novel method that leverages large language models (LLMs) to deduce causal relationships in general causal graph recovery tasks.
  arXiv  Detail & Related papers  (2024-02-23T13:02:10Z)
• Discovery of the Hidden World with Large Language Models [95.58823685009727]
  This paper presents Causal representatiOn AssistanT (COAT) that introduces large language models (LLMs) to bridge the gap. LLMs are trained on massive observations of the world and have demonstrated great capability in extracting key information from unstructured data. COAT also adopts CDs to find causal relations among the identified variables as well as to provide feedback to LLMs to iteratively refine the proposed factors.
  arXiv  Detail & Related papers  (2024-02-06T12:18:54Z)
• Multi-modal Causal Structure Learning and Root Cause Analysis [67.67578590390907]
  We propose Mulan, a unified multi-modal causal structure learning method for root cause localization. We leverage a log-tailored language model to facilitate log representation learning, converting log sequences into time-series data. We also introduce a novel key performance indicator-aware attention mechanism for assessing modality reliability and co-learning a final causal graph.
  arXiv  Detail & Related papers  (2024-02-04T05:50:38Z)
• CCLF: A Contrastive-Curiosity-Driven Learning Framework for Sample-Efficient Reinforcement Learning [56.20123080771364]
  We develop a model-agnostic Contrastive-Curiosity-Driven Learning Framework (CCLF) for reinforcement learning. CCLF fully exploit sample importance and improve learning efficiency in a self-supervised manner. We evaluate this approach on the DeepMind Control Suite, Atari, and MiniGrid benchmarks.
  arXiv  Detail & Related papers  (2022-05-02T14:42:05Z)
• Confounder Identification-free Causal Visual Feature Learning [84.28462256571822]
  We propose a novel Confounder Identification-free Causal Visual Feature Learning (CICF) method, which obviates the need for identifying confounders. CICF models the interventions among different samples based on front-door criterion, and then approximates the global-scope intervening effect upon the instance-level interventions. We uncover the relation between CICF and the popular meta-learning strategy MAML, and provide an interpretation of why MAML works from the theoretical perspective.
  arXiv  Detail & Related papers  (2021-11-26T10:57:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.