Benchmarking Vision-Language Models for Automated Pathology Diagnosis and Report Generation

• URL: http://arxiv.org/abs/2609.00866v1
• Date: Tue, 01 Sep 2026 08:01:31 GMT
• Title: Benchmarking Vision-Language Models for Automated Pathology Diagnosis and Report Generation
• Abstract summary: We introduce a clinically curated Pan-Asia WSI--report dataset of approximately 10,500 pairs from five institutions.<n>We establish the REG 2025 benchmark through a MICCAI challenge for systematic evaluation of multimodal models.
• Score: 28.553483401938195
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The rapid advancement of vision-language models (VLMs) has accelerated progress in computational pathology; however, whole-slide image (WSI)-based pathology report generation remains limited by the scarcity of large-scale WSI--report datasets and the complexity of mapping spatially distributed visual patterns to structured clinical text. To address this, we introduce a clinically curated Pan-Asia WSI--report dataset of approximately 10,500 pairs from five institutions and establish the REG 2025 benchmark through a MICCAI challenge for systematic evaluation of multimodal models. We analyze submitted methods spanning pretrained VLMs, multiple-instance learning frameworks, hierarchical expert models, retrieval-augmented generation, and cross-modal Transformers. Rather than indicating that VLM use alone was sufficient for superior performance, the results suggest that top-performing methods benefited from structured report representations, hierarchical diagnostic decomposition, and effective multimodal grounding. We identify key limitations, including instability in quantitative attribute estimation (e.g., numeric hallucination) and a tendency toward diagnostic overspecification, with some errors resembling known diagnostic pitfalls in routine pathology. These findings establish REG 2025 as a benchmark for evaluating WSI-based structured report generation and vision-language understanding in computational pathology, providing insights for the design of clinically grounded multimodal pathology models.

Related papers

• Concept-Grounded Reasoning with Prompt-Driven Localization for Interpretable Structured Report Generation [66.62662330558906]
  CORAL is a multimodal framework that integrates spatial grounding and concept-level supervision into a unified reasoning process.<n>CORAL employs a prompt-driven medical segmentation model to localize lesions and predicts multi-class clinical attributes through a Concept Bottleneck module.<n>Experiments on BUS-CoT and IU X-ray datasets demonstrate consistent improvements in diagnostic accuracy, concept consistency, and report quality over strong general-purpose and medical MLLMs.
  arXiv  Detail & Related papers  (2026-09-14T10:21:23Z)
• Paired Uterine Whole-Slide Images and Pathology Reports for Multimodal Computational Pathology [54.42160154368719]
  Whole-slide images (WSI) have enabled the digital transformation of pathology.<n> multimodal models that jointly analyze histopathology images and pathology reports have shown promising potential for automated pathology report generation and AI-assisted diagnosis.<n>Here, we introduce TUM-Uteria, a uterine pathology dataset comprising WSIs paired with diagnostic pathology reports at both the case and slide levels.
  arXiv  Detail & Related papers  (2026-07-04T20:34:59Z)
• Multimodal Graph-based Classification of Esophageal Motility Disorders [73.90451172929117]
  Diagnosing esophageal motility disorders pose significant challenges due to the complexity of high-resolution impedance manometry data and variability in clinical interpretation.<n>This work explores the feasibility of a multimodal Machine Learning-based classification approach that combines HRIM recordings with patient-specific information and incorporates a graph-based modeling of esophageal physiology.
  arXiv  Detail & Related papers  (2026-05-13T14:52:12Z)
• Clinical Cognition Alignment for Gastrointestinal Diagnosis with Multimodal LLMs [63.535652574541764]
  Multimodal Large Language Models (MLLMs) have demonstrated remarkable potential in medical image analysis.<n>Their application in gastrointestinal endoscopy is currently hindered by two critical limitations.<n>We propose a novel Clinical-Cognitive-Aligned (CogAlign) framework to address these challenges.
  arXiv  Detail & Related papers  (2026-03-21T07:47:37Z)
• A Semantically Enhanced Generative Foundation Model Improves Pathological Image Synthesis [82.01597026329158]
  We introduce a Correlation-Regulated Alignment Framework for Tissue Synthesis (CRAFTS) for pathology-specific text-to-image synthesis.<n>CRAFTS incorporates a novel alignment mechanism that suppresses semantic drift to ensure biological accuracy.<n>This model generates diverse pathological images spanning 30 cancer types, with quality rigorously validated by objective metrics and pathologist evaluations.
  arXiv  Detail & Related papers  (2025-12-15T10:22:43Z)
• PathoHR: Hierarchical Reasoning for Vision-Language Models in Pathology [3.459714932882085]
  Current vision-language (VL) models often struggle to capture the complex reasoning required for interpreting structured pathological reports.<n>We propose PathoHR-Bench, a novel benchmark designed to evaluate VL models' abilities in hierarchical semantic understanding and compositional reasoning within the pathology domain.<n>We further introduce a pathology-specific VL training scheme that generates enhanced and perturbed samples for multimodal contrastive learning.
  arXiv  Detail & Related papers  (2025-09-07T15:42:38Z)
• AMRG: Extend Vision Language Models for Automatic Mammography Report Generation [4.366802575084445]
  Mammography report generation is a critical yet underexplored task in medical AI.<n>We introduce AMRG, the first end-to-end framework for generating narrative mammography reports.<n>We train and evaluate AMRG on DMID, a publicly available dataset of paired high-resolution mammograms and diagnostic reports.
  arXiv  Detail & Related papers  (2025-08-12T06:37:41Z)
• DiagR1: A Vision-Language Model Trained via Reinforcement Learning for Digestive Pathology Diagnosis [7.5173141954286775]
  We construct a large scale gastrointestinal pathology dataset containing both microscopic descriptions and diagnostic conclusions.<n>This design guides the model to better capture image specific features and maintain semantic consistency in generation.<n>Our solution outperforms state of the art models with 18.7% higher clinical relevance, 32.4% improved structural completeness, and 41.2% fewer diagnostic errors.
  arXiv  Detail & Related papers  (2025-07-24T14:12:20Z)
• Clinical-grade Multi-Organ Pathology Report Generation for Multi-scale Whole Slide Images via a Semantically Guided Medical Text Foundation Model [3.356716093747221]
  We propose a novel Patient-level Multi-organ Pathology Report Generation (PMPRG) model to generate pathology reports for patients. Our model achieved a METEOR score of 0.68, demonstrating the effectiveness of our approach.
  arXiv  Detail & Related papers  (2024-09-23T22:22:32Z)
• PathInsight: Instruction Tuning of Multimodal Datasets and Models for Intelligence Assisted Diagnosis in Histopathology [7.87900104748629]
  We have meticulously compiled a dataset of approximately 45,000 cases, covering over 6 different tasks. We have fine-tuned multimodal large models, specifically LLaVA, Qwen-VL, InternLM, with this dataset to enhance instruction-based performance.
  arXiv  Detail & Related papers  (2024-08-13T17:05:06Z)
• Potential of Multimodal Large Language Models for Data Mining of Medical Images and Free-text Reports [51.45762396192655]
  Multimodal large language models (MLLMs) have recently transformed many domains, significantly affecting the medical field. Notably, Gemini-Vision-series (Gemini) and GPT-4-series (GPT-4) models have epitomized a paradigm shift in Artificial General Intelligence for computer vision. This study evaluated the performance of the Gemini, GPT-4, and 4 popular large models for an exhaustive evaluation across 14 medical imaging datasets.
  arXiv  Detail & Related papers  (2024-07-08T09:08:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.