A Multi-modal Agentic Co-pilot for Evidence Grounded Computational Pathology

• URL: http://arxiv.org/abs/2606.08093v1
• Date: Sat, 06 Jun 2026 10:36:30 GMT
• Title: A Multi-modal Agentic Co-pilot for Evidence Grounded Computational Pathology
• Abstract summary: We present PathPocket, a multimodal AI agentic co-pilot designed specifically for evidence grounded pathology.<n>We construct the most comprehensive pathology evidence corpus to date, encompassing approximately 110,472 public and authorized documents structured across a rigorous hierarchy of evidence from clinical guideline to expert opinion.<n>This hypergraph provides traceable evidence for a collaborative multi-agent reasoning framework integrating input understanding, evidence retrieval, filtering, and diagnosis generation.<n>We rigorously evaluate the system on a multidimensional benchmark of over 200,000 real-world cases, where it significantly outperforms existing state-of-the-arts.
• Score: 27.958607954013036
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Pathology is the cornerstone of modern medicine, where accurate decision-making relies heavily on evidence-based practices. While artificial intelligence (AI) has the potential to transform clinical workflows, the intersection of AI and evidence-based medicine remains under-explored, with primitive attempts restricted to text-only general medicine. In this work, we present PathPocket, a multimodal AI agentic co-pilot designed specifically for evidence grounded pathology. We construct the most comprehensive pathology evidence corpus to date, encompassing approximately 110,472 public and authorized documents structured across a rigorous hierarchy of evidence from clinical guideline to expert opinion. From this meticulously graded foundation, we build a large-scale multimodal pathology hypergraph containing over 4.55 million entities and 7.10 million relations. Serving as a robust knowledge engine, this hypergraph provides traceable evidence for a collaborative multi-agent reasoning framework integrating input understanding, evidence retrieval, filtering, and diagnosis generation. This enables PathPocket to seamlessly resolve a wide spectrum of clinical tasks, ranging from text-only queries to complex multimodal diagnostics involving region-of-interest (ROI) and gigapixel whole-slide images (WSIs). We rigorously evaluate the system on a multidimensional benchmark of over 200,000 real-world cases, where it significantly outperforms existing state-of-the-arts. Crucially, extensive user studies demonstrate that PathPocket substantially improves the diagnostic accuracy and confidence of pathologists. By directly grounding pathology interpretations in verifiable literature, PathPocket offers a practical and scalable solution for the future of evidence grounded computational pathology.

Related papers

• MedReaMM: Evaluating Large Multimodal Models on Expert-Level Clinical Diagnostic Synthesis [61.266743965495884]
  We introduce MedReaMM, a benchmark designed to evaluate models' ability to synthesize heterogeneous clinical evidence.<n>MedReaMM comprises 625 expert-validated cases with an average of 2.79 medical images per case and a total of 1,042 standardized diagnoses annotated with ICD-11 codes.<n>We evaluate 23 Large Multimodal Models (LMMs) and find that most achieve diagnostic accuracy scores below 50%, underscoring a substantial gap in multimodal diagnostic synthesis capability.
  arXiv  Detail & Related papers  (2026-08-23T09:37:53Z)
• Social Chain of Thought: A Multi-Agent Architecture Grounded in Medical Differential Diagnosis Methodology [21.116270925609477]
  Medical diagnostic reasoning is a high-impact use case for LLMs.<n>OpenAI reports that more than 5% of ChatGPT messages globally are healthcare-related.<n>We introduce Social Chain of Thought (SCoT),a multi-round pipeline for medical differential diagnosis.
  arXiv  Detail & Related papers  (2026-08-11T20:38:03Z)
• MultiDx: A Multi-Source Knowledge Integration Framework towards Diagnostic Reasoning [66.94527468532843]
  We propose MultiDx, a two-stage diagnostic reasoning framework.<n>It first generates suspected diagnoses and reasoning paths by leveraging knowledge from web search, SOAP-formatted case, and clinical case database.<n>Then it integrates multi-perspective evidence through matching, voting, and differential diagnosis to generate the final prediction.
  arXiv  Detail & Related papers  (2026-04-27T08:46:29Z)
• Multi-Agent Intelligence for Multidisciplinary Decision-Making in Gastrointestinal Oncology [13.663415863327996]
  A hierarchical Multi-Agent Framework is proposed, which emulates the collaborative workflow of a human Multidisciplinary Team (MDT)<n>The system attained a composite expert evaluation score of 4.60/5.00, thereby demonstrating a substantial improvement over the monolithic baseline.<n>The findings indicate that mimetic, agent-based collaboration provides a scalable, interpretable, and clinically robust paradigm for automated decision support in oncology.
  arXiv  Detail & Related papers  (2025-12-09T14:56:40Z)
• TeamPath: Building MultiModal Pathology Experts with Reasoning AI Copilots [32.22233464805693]
  We introduce TeamPath, an AI system powered by reinforcement learning and router-enhanced solutions based on large-scale histopathology datasets.<n>TeamPath works as a virtual assistant for expert-level disease diagnosis, patch-level information summarization, and cross-modality generation.<n>We collaborate with pathologists from Yale School of Medicine to demonstrate that TeamPath can assist them in working more efficiently.
  arXiv  Detail & Related papers  (2025-11-20T13:29:35Z)
• RAD: Towards Trustworthy Retrieval-Augmented Multi-modal Clinical Diagnosis [56.373297358647655]
  Retrieval-Augmented Diagnosis (RAD) is a novel framework that injects external knowledge into multimodal models directly on downstream tasks.<n>RAD operates through three key mechanisms: retrieval and refinement of disease-centered knowledge from multiple medical sources, a guideline-enhanced contrastive loss transformer, and a dual decoder.
  arXiv  Detail & Related papers  (2025-09-24T10:36:14Z)
• End-to-End Agentic RAG System Training for Traceable Diagnostic Reasoning [52.12425911708585]
  Deep-DxSearch is an agentic RAG system trained end-to-end with reinforcement learning (RL)<n>In Deep-DxSearch, we first construct a large-scale medical retrieval corpus comprising patient records and reliable medical knowledge sources.<n> Experiments demonstrate that our end-to-end RL training framework consistently outperforms prompt-engineering and training-free RAG approaches.
  arXiv  Detail & Related papers  (2025-08-21T17:42:47Z)
• Evidence-based diagnostic reasoning with multi-agent copilot for human pathology [7.976907866539546]
  Current multimodal large language models (MLLMs) in computational pathology face limitations.<n>We introduce PathChat+, a new MLLM specifically designed for human pathology, trained on over 1 million diverse, pathology-specific instruction samples.<n>We also present SlideSeek, a reasoning-enabled multi-agent AI system leveraging PathChat+ to autonomously evaluate gigapixel whole-slide images.
  arXiv  Detail & Related papers  (2025-06-26T03:02:16Z)
• RadFabric: Agentic AI System with Reasoning Capability for Radiology [61.25593938175618]
  RadFabric is a multi agent, multimodal reasoning framework that unifies visual and textual analysis for comprehensive CXR interpretation.<n>System employs specialized CXR agents for pathology detection, an Anatomical Interpretation Agent to map visual findings to precise anatomical structures, and a Reasoning Agent powered by large multimodal reasoning models to synthesize visual, anatomical, and clinical data into transparent and evidence based diagnoses.
  arXiv  Detail & Related papers  (2025-06-17T03:10:33Z)
• PathFinder: A Multi-Modal Multi-Agent System for Medical Diagnostic Decision-Making Applied to Histopathology [11.366456922386263]
  Diagnosing diseases through whole slide images is fundamental in modern pathology.<n>Traditional AI approaches fail short of such a holistic, iterative, multi-scale diagnostic procedure.<n>We introduce PathFinder, a multi-modal, multi-agent framework that emulates the decision-making process of expert pathologists.
  arXiv  Detail & Related papers  (2025-02-13T03:08:02Z)
• Large-scale Long-tailed Disease Diagnosis on Radiology Images [51.453990034460304]
  RadDiag is a foundational model supporting 2D and 3D inputs across various modalities and anatomies. Our dataset, RP3D-DiagDS, contains 40,936 cases with 195,010 scans covering 5,568 disorders.
  arXiv  Detail & Related papers  (2023-12-26T18:20:48Z)
• Validating polyp and instrument segmentation methods in colonoscopy through Medico 2020 and MedAI 2021 Challenges [58.32937972322058]
  "Medico automatic polyp segmentation (Medico 2020)" and "MedAI: Transparency in Medical Image (MedAI 2021)" competitions. We present a comprehensive summary and analyze each contribution, highlight the strength of the best-performing methods, and discuss the possibility of clinical translations of such methods into the clinic.
  arXiv  Detail & Related papers  (2023-07-30T16:08:45Z)
• PathAsst: A Generative Foundation AI Assistant Towards Artificial General Intelligence of Pathology [15.419350834457136]
  We present PathAsst, a multimodal generative foundation AI assistant to revolutionize diagnostic and predictive analytics in pathology. The development of PathAsst involves three pivotal steps: data acquisition, CLIP model adaptation, and the training of PathAsst's multimodal generative capabilities. The experimental results of PathAsst show the potential of harnessing AI-powered generative foundation model to improve pathology diagnosis and treatment processes.
  arXiv  Detail & Related papers  (2023-05-24T11:55:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.