Exploring Multi-Modal Data with Tool-Augmented LLM Agents for Precise Causal Discovery

• URL: http://arxiv.org/abs/2412.13667v2
• Date: Sat, 31 May 2025 19:01:54 GMT
• Title: Exploring Multi-Modal Data with Tool-Augmented LLM Agents for Precise Causal Discovery
• Authors: ChengAo Shen, Zhengzhang Chen, Dongsheng Luo, Dongkuan Xu, Haifeng Chen, Jingchao Ni,
• Abstract summary: We introduce MATMCD, a multi-agent system powered by tool-augmented LLMs.<n>Our empirical study suggests the significant potential of multi-modality enhanced causal discovery.
• Score: 45.777770849667775
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Causal discovery is an imperative foundation for decision-making across domains, such as smart health, AI for drug discovery and AIOps. Traditional statistical causal discovery methods, while well-established, predominantly rely on observational data and often overlook the semantic cues inherent in cause-and-effect relationships. The advent of Large Language Models (LLMs) has ushered in an affordable way of leveraging the semantic cues for knowledge-driven causal discovery, but the development of LLMs for causal discovery lags behind other areas, particularly in the exploration of multi-modal data. To bridge the gap, we introduce MATMCD, a multi-agent system powered by tool-augmented LLMs. MATMCD has two key agents: a Data Augmentation agent that retrieves and processes modality-augmented data, and a Causal Constraint agent that integrates multi-modal data for knowledge-driven reasoning. The proposed design of the inner-workings ensures successful cooperation of the agents. Our empirical study across seven datasets suggests the significant potential of multi-modality enhanced causal discovery.

Related papers

• Structured and Abstractive Reasoning on Multi-modal Relational Knowledge Images [58.553448128258566]
  This paper bridges the dual gaps in large-scale high-quality data and capability enhancement methodologies.<n>We introduce STAR-64K, a dataset comprising 64K high-quality multi-modal instruction samples, and conduct experiments across 5 open-source MLLMs.
  arXiv  Detail & Related papers  (2025-10-22T02:23:40Z)
• Revealing Multimodal Causality with Large Language Models [80.95511545591107]
  We propose MLLM-CD, a novel framework for multimodal causal discovery from unstructured data.<n>It consists of three key components: (1) a novel contrastive factor discovery module to identify genuine multimodal factors; (2) a statistical causal structure discovery module to infer causal relationships among discovered factors; and (3) an iterative multimodal counterfactual reasoning module to refine the discovery outcomes.<n>Extensive experiments on both synthetic and real-world datasets demonstrate the effectiveness of the proposed MLLM-CD.
  arXiv  Detail & Related papers  (2025-09-22T13:45:17Z)
• Causal MAS: A Survey of Large Language Model Architectures for Discovery and Effect Estimation [5.062951330356307]
  Large Language Models (LLMs) have demonstrated remarkable capabilities in various reasoning and generation tasks.<n>Their proficiency in complex causal reasoning, discovery, and estimation remains an area of active development.<n>Multi-agent systems, leveraging the collaborative or specialized abilities of multiple LLM-based agents, are emerging as a powerful paradigm to address these limitations.
  arXiv  Detail & Related papers  (2025-08-31T20:48:31Z)
• Anomaly Detection and Generation with Diffusion Models: A Survey [51.61574868316922]
  Anomaly detection (AD) plays a pivotal role across diverse domains, including cybersecurity, finance, healthcare, and industrial manufacturing.<n>Recent advancements in deep learning, specifically diffusion models (DMs), have sparked significant interest.<n>This survey aims to guide researchers and practitioners in leveraging DMs for innovative AD solutions across diverse applications.
  arXiv  Detail & Related papers  (2025-06-11T03:29:18Z)
• Beyond Correlation: Towards Causal Large Language Model Agents in Biomedicine [0.14700417433722487]
  Large Language Models (LLMs) show promise in biomedicine but lack true causal understanding, relying instead on correlations.<n>This paper envisions causal LLM agents that integrate multimodal data (text, images, genomics, etc.) and perform intervention-based reasoning to infer cause-and-effect.
  arXiv  Detail & Related papers  (2025-05-22T17:52:59Z)
• Can Large Language Models Help Experimental Design for Causal Discovery? [94.66802142727883]
  Large Language Model Guided Intervention Targeting (LeGIT) is a robust framework that effectively incorporates LLMs to augment existing numerical approaches for the intervention targeting in causal discovery. LeGIT demonstrates significant improvements and robustness over existing methods and even surpasses humans.
  arXiv  Detail & Related papers  (2025-03-03T03:43:05Z)
• RAG-Enhanced Collaborative LLM Agents for Drug Discovery [28.025359322895905]
  CLADD is a retrieval-augmented generation (RAG)-empowered agentic system tailored to drug discovery tasks.<n>We show that it outperforms general-purpose and domain-specific LLMs as well as traditional deep learning approaches.
  arXiv  Detail & Related papers  (2025-02-22T00:12:52Z)
• Regularized Multi-LLMs Collaboration for Enhanced Score-based Causal Discovery [13.654021365091305]
  We explore the potential of using large language models (LLMs) to enhance causal discovery approaches.<n>We propose a general framework to utilise the capacity of not only one but multiple LLMs to augment the discovery process.
  arXiv  Detail & Related papers  (2024-11-27T01:56:21Z)
• Online Multi-modal Root Cause Analysis [61.94987309148539]
  Root Cause Analysis (RCA) is essential for pinpointing the root causes of failures in microservice systems. Existing online RCA methods handle only single-modal data overlooking, complex interactions in multi-modal systems. We introduce OCEAN, a novel online multi-modal causal structure learning method for root cause localization.
  arXiv  Detail & Related papers  (2024-10-13T21:47:36Z)
• From Pre-training Corpora to Large Language Models: What Factors Influence LLM Performance in Causal Discovery Tasks? [51.42906577386907]
  This study explores the factors influencing the performance of Large Language Models (LLMs) in causal discovery tasks. A higher frequency of causal mentions correlates with better model performance, suggesting that extensive exposure to causal information during training enhances the models' causal discovery capabilities.
  arXiv  Detail & Related papers  (2024-07-29T01:45:05Z)
• Multi-Agent Causal Discovery Using Large Language Models [10.020595983728482]
  Large Language Models (LLMs) have demonstrated significant potential in causal discovery tasks. This paper introduces a general framework to investigate this potential. Our proposed framework shows promising results by effectively utilizing LLMs expert knowledge, reasoning capabilities, multi-agent cooperation, and statistical causal methods.
  arXiv  Detail & Related papers  (2024-07-21T06:21:47Z)
• RealTCD: Temporal Causal Discovery from Interventional Data with Large Language Model [15.416325455014462]
  Temporal causal discovery aims to identify temporal causal relationships between variables directly from observations. Existing methods mainly focus on synthetic datasets with heavy reliance on intervention targets. We propose the RealTCD framework, which is able to leverage domain knowledge to discover temporal causal relationships without interventional targets.
  arXiv  Detail & Related papers  (2024-04-23T06:52:40Z)
• Large Language Models for Causal Discovery: Current Landscape and Future Directions [5.540272236593385]
  Causal discovery (CD) and Large Language Models (LLMs) have emerged as transformative fields in artificial intelligence. This survey examines how LLMs are transforming CD across three key dimensions: direct causal extraction from text, integration of domain knowledge into statistical methods, and refinement of causal structures.
  arXiv  Detail & Related papers  (2024-02-16T20:48:53Z)
• Discovery of the Hidden World with Large Language Models [95.58823685009727]
  This paper presents Causal representatiOn AssistanT (COAT) that introduces large language models (LLMs) to bridge the gap. LLMs are trained on massive observations of the world and have demonstrated great capability in extracting key information from unstructured data. COAT also adopts CDs to find causal relations among the identified variables as well as to provide feedback to LLMs to iteratively refine the proposed factors.
  arXiv  Detail & Related papers  (2024-02-06T12:18:54Z)
• Multi-modal Causal Structure Learning and Root Cause Analysis [67.67578590390907]
  We propose Mulan, a unified multi-modal causal structure learning method for root cause localization. We leverage a log-tailored language model to facilitate log representation learning, converting log sequences into time-series data. We also introduce a novel key performance indicator-aware attention mechanism for assessing modality reliability and co-learning a final causal graph.
  arXiv  Detail & Related papers  (2024-02-04T05:50:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.