Improving Clinical Diagnosis with Counterfactual Multi-Agent Reasoning

• URL: http://arxiv.org/abs/2603.27820v1
• Date: Sun, 29 Mar 2026 19:14:26 GMT
• Title: Improving Clinical Diagnosis with Counterfactual Multi-Agent Reasoning
• Abstract summary: Clinical diagnosis is a complex reasoning process in which clinicians gather evidence, form hypotheses, and test them against alternative explanations.<n>We propose a counterfactual multi-agent diagnostic framework inspired by clinician training that makes hypothesis testing explicit and evidence-grounded.
• Score: 16.709358805587506
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Clinical diagnosis is a complex reasoning process in which clinicians gather evidence, form hypotheses, and test them against alternative explanations. In medical training, this reasoning is explicitly developed through counterfactual questioning--e.g., asking how a diagnosis would change if a key symptom were absent or altered--to strengthen differential diagnosis skills. As large language model (LLM)-based systems are increasingly used for diagnostic support, ensuring the interpretability of their recommendations becomes critical. However, most existing LLM-based diagnostic agents reason over fixed clinical evidence without explicitly testing how individual findings support or weaken competing diagnoses. In this work, we propose a counterfactual multi-agent diagnostic framework inspired by clinician training that makes hypothesis testing explicit and evidence-grounded. Our framework introduces counterfactual case editing to modify clinical findings and evaluate how these changes affect competing diagnoses. We further define the Counterfactual Probability Gap, a method that quantifies how strongly individual findings support a diagnosis by measuring confidence shifts under these edits. These counterfactual signals guide multi-round specialist discussions, enabling agents to challenge unsupported hypotheses, refine differential diagnoses, and produce more interpretable reasoning trajectories. Across three diagnostic benchmarks and seven LLMs, our method consistently improves diagnostic accuracy over prompting and prior multi-agent baselines, with the largest gains observed in complex and ambiguous cases. Human evaluation further indicates that our framework produces more clinically useful, reliable, and coherent reasoning. These results suggest that incorporating counterfactual evidence verification is an important step toward building reliable AI systems for clinical decision support.

Related papers

• EviDx: Evidence-Aware Active Diagnosis with Scaffolded LLM Agents [13.21048121687754]
  We introduce EviDx, an evidence-aware active diagnosis framework.<n>It pairs patient-specific diagnostic environments with a clinical diagnostic scaffold and an observer-guided runtime harness.<n>EviDx improves diagnostic performance and process stability while revealing model-dependent capability boundaries.
  arXiv  Detail & Related papers  (2026-08-25T13:54:30Z)
• Evaluating Multi-Turn Multimodal Diagnostic Reasoning on Challenging Real-World Clinical Cases [59.5328455571863]
  ClinMM-Bench is the largest multi-turn multimodal clinical diagnostic evaluation benchmark to date.<n>It contains 1,089 challenging real-world clinical cases and 3,760 medical images across eight specialties.<n> proprietary models achieved the highest overall diagnostic accuracy, but the proportion of completely correct diagnoses remained limited.
  arXiv  Detail & Related papers  (2026-07-28T16:19:03Z)
• MAGIS: Evidence-Based Multi-Agent Reasoning for Interpretable Strabismus Clinical Decision-Making [18.234379859242406]
  Strabismus is a common ocular disorder that requires fine-grained subtype diagnosis for individualized treatment planning.<n>We propose MAGIS, an evidence-based Multi-AGent reasoning for Interpretable Strabismus diagnosis framework.
  arXiv  Detail & Related papers  (2026-06-08T09:21:53Z)
• Medical Model Synthesis Architectures: A Case Study [72.46211022258122]
  Current AI systems struggle with calibrated reasoning under uncertainty.<n>We propose a framework for AI systems that can make practically useful but formally transparent clinical predictions under uncertainty.<n>We show how an initial proof-of-concept of this framework can be used for differential diagnosis.
  arXiv  Detail & Related papers  (2026-05-10T19:30:16Z)
• MultiDx: A Multi-Source Knowledge Integration Framework towards Diagnostic Reasoning [66.94527468532843]
  We propose MultiDx, a two-stage diagnostic reasoning framework.<n>It first generates suspected diagnoses and reasoning paths by leveraging knowledge from web search, SOAP-formatted case, and clinical case database.<n>Then it integrates multi-perspective evidence through matching, voting, and differential diagnosis to generate the final prediction.
  arXiv  Detail & Related papers  (2026-04-27T08:46:29Z)
• CXReasonAgent: Evidence-Grounded Diagnostic Reasoning Agent for Chest X-rays [16.09644321057171]
  Chest X-ray plays a central role in thoracic diagnosis, and its interpretation inherently requires multi-step, evidence-grounded reasoning.<n>Large vision-language models (LVLMs) generate plausible responses that are not faithfully grounded in diagnostic evidence.<n>We present CXReasonAgent, a diagnostic agent that integrates a large language model with clinically grounded diagnostic tools.
  arXiv  Detail & Related papers  (2026-02-26T17:51:21Z)
• ClinDEF: A Dynamic Evaluation Framework for Large Language Models in Clinical Reasoning [58.01333341218153]
  We propose ClinDEF, a dynamic framework for assessing clinical reasoning in LLMs through simulated diagnostic dialogues.<n>Our method generates patient cases and facilitates multi-turn interactions between an LLM-based doctor and an automated patient agent.<n>Experiments show that ClinDEF effectively exposes critical clinical reasoning gaps in state-of-the-art LLMs.
  arXiv  Detail & Related papers  (2025-12-29T12:58:58Z)
• Timely Clinical Diagnosis through Active Test Selection [49.091903570068155]
  We propose ACTMED (Adaptive Clinical Test selection via Model-based Experimental Design) to better emulate real-world diagnostic reasoning.<n>LLMs act as flexible simulators, generating plausible patient state distributions and supporting belief updates without requiring structured, task-specific training data.<n>We evaluate ACTMED on real-world datasets and show it can optimize test selection to improve diagnostic accuracy, interpretability, and resource use.
  arXiv  Detail & Related papers  (2025-10-21T18:10:45Z)
• Simulating Viva Voce Examinations to Evaluate Clinical Reasoning in Large Language Models [51.91760712805404]
  We introduce VivaBench, a benchmark for evaluating sequential clinical reasoning in large language models (LLMs)<n>Our dataset consists of 1762 physician-curated clinical vignettes structured as interactive scenarios that simulate a (oral) examination in medical training.<n>Our analysis identified several failure modes that mirror common cognitive errors in clinical practice.
  arXiv  Detail & Related papers  (2025-10-11T16:24:35Z)
• End-to-End Agentic RAG System Training for Traceable Diagnostic Reasoning [52.12425911708585]
  Deep-DxSearch is an agentic RAG system trained end-to-end with reinforcement learning (RL)<n>In Deep-DxSearch, we first construct a large-scale medical retrieval corpus comprising patient records and reliable medical knowledge sources.<n> Experiments demonstrate that our end-to-end RL training framework consistently outperforms prompt-engineering and training-free RAG approaches.
  arXiv  Detail & Related papers  (2025-08-21T17:42:47Z)
• LLM-Driven Medical Document Analysis: Enhancing Trustworthy Pathology and Differential Diagnosis [13.435898630240416]
  We propose a trustworthy medical document analysis platform that fine-tunes a LLaMA-v3 using low-rank adaptation.<n>Our approach utilizes DDXPlus, the largest benchmark dataset for differential diagnosis.<n>The developed web-based platform allows users to submit their own unstructured medical documents and receive accurate, explainable diagnostic results.
  arXiv  Detail & Related papers  (2025-06-24T15:12:42Z)
• MSDiagnosis: A Benchmark for Evaluating Large Language Models in Multi-Step Clinical Diagnosis [8.641421154025211]
  We propose a Chinese clinical diagnostic benchmark, called MSDiagnosis.<n>This benchmark consists of 2,225 cases from 12 departments, covering tasks such as primary diagnosis, differential diagnosis, and final diagnosis.
  arXiv  Detail & Related papers  (2024-08-19T14:31:57Z)
• Medical Dialogue Generation via Intuitive-then-Analytical Differential Diagnosis [14.17497921394565]
  We propose a medical dialogue generation framework with the Intuitive-then-Analytic Differential Diagnosis (IADDx) Our method starts with a differential diagnosis via retrieval-based intuitive association and subsequently refines it through a graph-enhanced analytic procedure. Experimental results on two datasets validate the efficacy of our method.
  arXiv  Detail & Related papers  (2024-01-12T12:35:19Z)
• Towards the Identifiability and Explainability for Personalized Learner Modeling: An Inductive Paradigm [36.60917255464867]
  We propose an identifiable cognitive diagnosis framework (ID-CDF) based on a novel response-proficiency-response paradigm inspired by encoder-decoder models. We show that ID-CDF can effectively address the problems without loss of diagnosis preciseness.
  arXiv  Detail & Related papers  (2023-09-01T07:18:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.