PathAgentBench: Benchmarking Evidence-Seeking Vision-Language Models on Whole-Slide Pathology Image

• URL: http://arxiv.org/abs/2607.19261v2
• Date: Thu, 30 Jul 2026 09:25:28 GMT
• Title: PathAgentBench: Benchmarking Evidence-Seeking Vision-Language Models on Whole-Slide Pathology Image
• Abstract summary: Whole-slide image (WSI) diagnosis requires identifying diagnostically relevant regions, examining them across magnifications, and integrating multi-scale evidence.<n>Most existing pathology benchmarks evaluate models on pre-cropped patches or pre-extracted slide features, leaving their ability to acquire evidence directly from gigapixel WSIs largely untested.<n>We introduce PathAgentBench, a benchmark for evaluating evidence-seeking vision-language models (VLMs) across four complementary capabilities.
• Score: 16.376573433449458
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Whole-slide image (WSI) diagnosis requires identifying diagnostically relevant regions, examining them across magnifications, and integrating multi-scale evidence. However, most existing pathology benchmarks evaluate models on pre-cropped patches or pre-extracted slide features, leaving their ability to acquire evidence directly from gigapixel WSIs largely untested. We introduce PathAgentBench, a benchmark for evaluating evidence-seeking vision-language models (VLMs) across four complementary capabilities: image-to-text matching for evidence interpretation, text-to-image retrieval for evidence verification, diagnostic-region localization for evidence acquisition, and multi-scale reasoning for evidence integration. The benchmark is organized as a diagnostic tree that links nested regions across magnifications with scale-specific findings and path-level diagnoses. It contains 1,822 TCGA WSIs and 17,135 diagnostic paths annotated by ten board-certified pathologists. An additional private cohort of 190 breast cancer WSIs with detailed annotations is used to evaluate autonomous whole-slide exploration. We evaluate 20 general-purpose, medical, and pathology-specialized models. Leading open-weight models achieve over 93% accuracy in multi-scale reasoning and over 50% accuracy in both cross-modal matching tasks. In contrast, diagnostic-region localization remains challenging: the best text-guided mean intersection-over-union is below 0.09, underperforming a simple center-based heuristic. During autonomous exploration, the unconditional hit rate decreases from 0.522 at low magnification to 0.185 at intermediate magnification and 0.020 at high magnification. These results reveal a pronounced gap between reasoning over curated evidence and acquiring that evidence directly from WSIs. PathAgentBench provides a unified framework for measuring and improving evidence-seeking pathology models.

Related papers

• MedReaMM: Evaluating Large Multimodal Models on Expert-Level Clinical Diagnostic Synthesis [61.266743965495884]
  We introduce MedReaMM, a benchmark designed to evaluate models' ability to synthesize heterogeneous clinical evidence.<n>MedReaMM comprises 625 expert-validated cases with an average of 2.79 medical images per case and a total of 1,042 standardized diagnoses annotated with ICD-11 codes.<n>We evaluate 23 Large Multimodal Models (LMMs) and find that most achieve diagnostic accuracy scores below 50%, underscoring a substantial gap in multimodal diagnostic synthesis capability.
  arXiv  Detail & Related papers  (2026-08-23T09:37:53Z)
• Agentic Visual Reasoning in Whole-Slide Pathology Images via Active Perception [30.71594045045451]
  Whole-slide visual reasoning requires identifying sparse diagnostic evidence in gigapixel slides.<n>We present AdaptivePath, an active-perception framework that formulates WSI evidence acquisition as sequential decision making.
  arXiv  Detail & Related papers  (2026-08-09T11:45:12Z)
• Enhancing Brain MRI Anomaly Detection and Reasoning with ROI Rethink and Synthetic Data [9.98520042320808]
  We present BrReMark, a framework that introduces explicit region marking into brain MRI diagnosis.<n>The model first generates hypotheses about potential abnormalities and grounds them through explicit bounding box marking.<n>We integrate a domain randomization-based pathology synthesis augmentation strategy to improve the model's generalizability to out-of-distribution (OOD) data.
  arXiv  Detail & Related papers  (2026-06-24T14:41:27Z)
• BenchX: Benchmarking AI Models for Cancer Detection and Localization with Demographic and Protocol Biases [59.19934491064188]
  Artificial intelligence (AI) has achieved remarkable success in medical imaging.<n>These models often perform inconsistently across real-world clinical settings.<n>We develop a benchmark of 85,355 CT scans that systematically evaluates 12 tumor-detection AI models.
  arXiv  Detail & Related papers  (2026-06-23T17:58:59Z)
• Anatomy-Aware Unsupervised Detection and Localization of Retinal Abnormalities in Optical Coherence Tomography [4.828654519487229]
  unsupervised anomaly detection framework learns the normative distribution of healthy retinal anatomy without lesion annotations.<n>We incorporate retinal layer-aware supervision and structured triplet learning to separate healthy from pathological representations.<n>On the Kermany dataset (AUROC: 0.799), our method substantially outperforms VAE, VQVAE, VQGAN, and f-AnoGAN baselines.
  arXiv  Detail & Related papers  (2026-04-24T01:09:05Z)
• Clinical Cognition Alignment for Gastrointestinal Diagnosis with Multimodal LLMs [63.535652574541764]
  Multimodal Large Language Models (MLLMs) have demonstrated remarkable potential in medical image analysis.<n>Their application in gastrointestinal endoscopy is currently hindered by two critical limitations.<n>We propose a novel Clinical-Cognitive-Aligned (CogAlign) framework to address these challenges.
  arXiv  Detail & Related papers  (2026-03-21T07:47:37Z)
• Beyond CLIP: Knowledge-Enhanced Multimodal Transformers for Cross-Modal Alignment in Diabetic Retinopathy Diagnosis [7.945705180020063]
  We propose a knowledge-enhanced joint embedding framework that integrates retinal fundus images, clinical text, and structured patient data.<n>Our framework achieves near-perfect text-to-image retrieval performance with Recall@1 of 99.94% compared to fine-tuned CLIP's 1.29%.
  arXiv  Detail & Related papers  (2025-12-22T18:41:45Z)
• DeepGI: Explainable Deep Learning for Gastrointestinal Image Classification [0.0]
  The study confronts common endoscopic challenges such as variable lighting, fluctuating camera angles, and frequent imaging artifacts.<n>The best performing models, VGG16 and MobileNetV2, each achieved a test accuracy of 96.5%.<n>The approach includes explainable AI via Grad-CAM visualization, enabling identification of image regions most influential to model predictions.
  arXiv  Detail & Related papers  (2025-11-26T22:35:57Z)
• An Explainable Hybrid AI Framework for Enhanced Tuberculosis and Symptom Detection [55.35661671061754]
  Tuberculosis remains a critical global health issue, particularly in resource-limited and remote areas.<n>We propose a framework which enhances disease and symptom detection on chest X-rays by integrating two supervised heads and a self-supervised head.<n>Our model achieves an accuracy of 98.85% for distinguishing between COVID-19, tuberculosis, and normal cases, and a macro-F1 score of 90.09% for multilabel symptom detection.
  arXiv  Detail & Related papers  (2025-10-21T17:18:55Z)
• Self-Supervised Anatomical Consistency Learning for Vision-Grounded Medical Report Generation [61.350584471060756]
  Vision-grounded medical report generation aims to produce clinically accurate descriptions of medical images.<n>We propose Self-Supervised Anatomical Consistency Learning (SS-ACL) to align generated reports with corresponding anatomical regions.<n>SS-ACL constructs a hierarchical anatomical graph inspired by the invariant top-down inclusion structure of human anatomy.
  arXiv  Detail & Related papers  (2025-09-30T08:59:06Z)
• RadFabric: Agentic AI System with Reasoning Capability for Radiology [61.25593938175618]
  RadFabric is a multi agent, multimodal reasoning framework that unifies visual and textual analysis for comprehensive CXR interpretation.<n>System employs specialized CXR agents for pathology detection, an Anatomical Interpretation Agent to map visual findings to precise anatomical structures, and a Reasoning Agent powered by large multimodal reasoning models to synthesize visual, anatomical, and clinical data into transparent and evidence based diagnoses.
  arXiv  Detail & Related papers  (2025-06-17T03:10:33Z)
• CPathAgent: An Agent-based Foundation Model for Interpretable High-Resolution Pathology Image Analysis Mimicking Pathologists' Diagnostic Logic [23.488576700623966]
  We introduce CPathAgent, an innovative agent-based approach that mimics pathologists' diagnostic workflow.<n>We develop a multi-stage training strategy that unifies patch-level, region-level, and WSI-level capabilities within a single model.<n>We construct PathMMU-HR2, the first expert-validated benchmark for large region analysis.
  arXiv  Detail & Related papers  (2025-05-26T20:22:19Z)
• Towards a Benchmark for Colorectal Cancer Segmentation in Endorectal Ultrasound Videos: Dataset and Model Development [59.74920439478643]
  In this paper, we collect and annotated the first benchmark dataset that covers diverse ERUS scenarios. Our ERUS-10K dataset comprises 77 videos and 10,000 high-resolution annotated frames. We introduce a benchmark model for colorectal cancer segmentation, named the Adaptive Sparse-context TRansformer (ASTR)
  arXiv  Detail & Related papers  (2024-08-19T15:04:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.