PathReasoning: A multimodal reasoning agent for query-based ROI navigation on whole-slide images

• URL: http://arxiv.org/abs/2511.21902v1
• Date: Wed, 26 Nov 2025 20:44:17 GMT
• Title: PathReasoning: A multimodal reasoning agent for query-based ROI navigation on whole-slide images
• Abstract summary: We propose "PathReasoning", a multi-modal reasoning agent that iteratively navigates across Whole Slide Images (WSIs)<n>PathReasoning builds a reasoning chain that gradually directs attention to diagnostically relevant areas.<n>It can substantially outperform strong ROI-selection approaches by 6.7% and 3.1% of AUROC on subtyping and longitudinal analysis tasks.
• Score: 12.145046046646215
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deciphering tumor microenvironment from Whole Slide Images (WSIs) is intriguing as it is key to cancer diagnosis, prognosis and treatment response. While these gigapixel images on one hand offer a comprehensive portrait of cancer, on the other hand, the extremely large size, as much as more than 10 billion pixels, make it challenging and time-consuming to navigate to corresponding regions to support diverse clinical inspection. Inspired by pathologists who conducted navigation on WSIs with a combination of sampling, reasoning and self-reflection, we proposed "PathReasoning", a multi-modal reasoning agent that iteratively navigates across WSIs through multiple rounds of reasoning and refinements. Specifically, starting with randomly sampled candidate regions, PathReasoning reviews current selections with self-reflection, reasoning over the correspondence between visual observations and clinical questions, and concludes by proposing new regions to explore. Across rounds, PathReasoning builds a reasoning chain that gradually directs attention to diagnostically relevant areas. PathReasoning turns each whole slide into a sequence of question-guided views, allowing the model to efficiently find informative ROIs within a fixed number of steps, without the need for dense pixel-level annotations. PathReasoning can substantially outperform strong ROI-selection approaches by 6.7% and 3.1% of AUROC on subtyping and longitudinal analysis tasks. The high-quality ROIs further support accurate report generation on breast cancer, significantly outperforming the standard GPT-4o by 10% in accuracy. PathReasoning prioritizes question-specific regions and constructs interpretable reasoning chains, supporting efficient slide review, consistent diagnostic interpretations, comprehensive reporting, and evidence traceability in digital pathology.

Related papers

• Agentic Visual Reasoning in Whole-Slide Pathology Images via Active Perception [30.71594045045451]
  Whole-slide visual reasoning requires identifying sparse diagnostic evidence in gigapixel slides.<n>We present AdaptivePath, an active-perception framework that formulates WSI evidence acquisition as sequential decision making.
  arXiv  Detail & Related papers  (2026-08-09T11:45:12Z)
• Pathologist Attention-Aligned Report Generation for Prostate Histopathology [28.438575541774487]
  We introduce human attention into the training of pathologist report generation models.<n>Experiments show average gains of 10.9% on NLP-based metrics and 19.3% in accuracy across five clinically relevant report components.
  arXiv  Detail & Related papers  (2026-07-21T23:08:20Z)
• PathAgentBench: Benchmarking Evidence-Seeking Vision-Language Models on Whole-Slide Pathology Image [16.376573433449458]
  Whole-slide image (WSI) diagnosis requires identifying diagnostically relevant regions, examining them across magnifications, and integrating multi-scale evidence.<n>Most existing pathology benchmarks evaluate models on pre-cropped patches or pre-extracted slide features, leaving their ability to acquire evidence directly from gigapixel WSIs largely untested.<n>We introduce PathAgentBench, a benchmark for evaluating evidence-seeking vision-language models (VLMs) across four complementary capabilities.
  arXiv  Detail & Related papers  (2026-07-21T16:33:44Z)
• Towards Clinically Interpretable Ophthalmic VQA via Spatially-Grounded Lesion Evidence [51.57735743553791]
  We introduce FundusGround, a new benchmark for clinically interpretable ophthalmic VQA with spatially-grounded lesion evidence.<n>Built upon this structured lesion evidence, 72,706 questions are then generated spanning four formats.<n>Experiments demonstrate that incorporating lesion-level visual evidence consistently improves model performance and transparency.
  arXiv  Detail & Related papers  (2026-05-21T12:37:03Z)
• Observe Less, Understand More: Cost-aware Cross-scale Observation for Remote Sensing Understanding [49.97682794425118]
  High-resolution (HR) imagery provides critical local details at much higher acquisition cost and limited coverage.<n>This motivates a cross-scale sensing strategy that selectively acquires HR imagery from LR-based global perception.<n>We present GL-10M, a large-scale benchmark of 10 million spatially aligned multi-resolution images.
  arXiv  Detail & Related papers  (2026-04-13T13:00:29Z)
• Act Like a Pathologist: Tissue-Aware Whole Slide Image Reasoning [21.809404751735503]
  We propose a question-guided, tissue-aware, and coarse-to-fine retrieval framework, HistoSelect.<n>Our approach outperforms existing methods and produces answers grounded in interpretable, pathologist-consistent regions.<n>Our results suggest that bringing human-like search and attention patterns into WSI reasoning is a promising direction for building practical and reliable pathology VLMs.
  arXiv  Detail & Related papers  (2026-02-28T14:22:53Z)
• MMedExpert-R1: Strengthening Multimodal Medical Reasoning via Domain-Specific Adaptation and Clinical Guideline Reinforcement [63.82954136824963]
  Medical Vision-Language Models excel at perception tasks with complex clinical reasoning required in real-world scenarios.<n>We propose a novel reasoning MedVLM that addresses these challenges through domain-specific adaptation and guideline reinforcement.
  arXiv  Detail & Related papers  (2026-01-16T02:32:07Z)
• PathMR: Multimodal Visual Reasoning for Interpretable Pathology Diagnosis [9.728322291979564]
  We propose PathMR, a cell-level Multimodal visual Reasoning framework for Pathological image analysis.<n>We show that PathMR consistently outperforms state-of-the-art visual reasoning methods in text generation quality, segmentation accuracy, and cross-modal alignment.
  arXiv  Detail & Related papers  (2025-08-28T14:46:24Z)
• Patho-AgenticRAG: Towards Multimodal Agentic Retrieval-Augmented Generation for Pathology VLMs via Reinforcement Learning [9.075284970935341]
  Patho-AgenticRAG is a database built on page-level embeddings from authoritative pathology textbooks.<n>It supports joint text-image search, enabling direct retrieval of textbook pages that contain both the queried text and relevant visual cues.<n>Patho-AgenticRAG significantly outperforms existing multimodal models in complex pathology tasks like multiple-choice diagnosis and visual question answering.
  arXiv  Detail & Related papers  (2025-08-04T10:03:08Z)
• RadFabric: Agentic AI System with Reasoning Capability for Radiology [61.25593938175618]
  RadFabric is a multi agent, multimodal reasoning framework that unifies visual and textual analysis for comprehensive CXR interpretation.<n>System employs specialized CXR agents for pathology detection, an Anatomical Interpretation Agent to map visual findings to precise anatomical structures, and a Reasoning Agent powered by large multimodal reasoning models to synthesize visual, anatomical, and clinical data into transparent and evidence based diagnoses.
  arXiv  Detail & Related papers  (2025-06-17T03:10:33Z)
• MIL vs. Aggregation: Evaluating Patient-Level Survival Prediction Strategies Using Graph-Based Learning [52.231128973251124]
  We compare various strategies for predicting survival at the WSI and patient level.<n>The former treats each WSI as an independent sample, mimicking the strategy adopted in other works.<n>The latter comprises methods to either aggregate the predictions of the several WSIs or automatically identify the most relevant slide.
  arXiv  Detail & Related papers  (2025-03-29T11:14:02Z)
• Towards a Benchmark for Colorectal Cancer Segmentation in Endorectal Ultrasound Videos: Dataset and Model Development [59.74920439478643]
  In this paper, we collect and annotated the first benchmark dataset that covers diverse ERUS scenarios. Our ERUS-10K dataset comprises 77 videos and 10,000 high-resolution annotated frames. We introduce a benchmark model for colorectal cancer segmentation, named the Adaptive Sparse-context TRansformer (ASTR)
  arXiv  Detail & Related papers  (2024-08-19T15:04:42Z)
• HistoGym: A Reinforcement Learning Environment for Histopathological Image Analysis [9.615399811006034]
  HistoGym aims to foster whole slide image diagnosis by mimicking the real-life processes of doctors. We offer various scenarios for different organs and cancers, including both WSI-based and selected region-based scenarios.
  arXiv  Detail & Related papers  (2024-08-16T17:19:07Z)
• WSSS4LUAD: Grand Challenge on Weakly-supervised Tissue Semantic Segmentation for Lung Adenocarcinoma [51.50991881342181]
  This challenge includes 10,091 patch-level annotations and over 130 million labeled pixels. First place team achieved mIoU of 0.8413 (tumor: 0.8389, stroma: 0.7931, normal: 0.8919)
  arXiv  Detail & Related papers  (2022-04-13T15:27:05Z)
• Act Like a Radiologist: Towards Reliable Multi-view Correspondence Reasoning for Mammogram Mass Detection [49.14070210387509]
  We propose an Anatomy-aware Graph convolutional Network (AGN) for mammogram mass detection. AGN is tailored for mammogram mass detection and endows existing detection methods with multi-view reasoning ability. Experiments on two standard benchmarks reveal that AGN significantly exceeds the state-of-the-art performance.
  arXiv  Detail & Related papers  (2021-05-21T06:48:34Z)
• Automated Prostate Cancer Diagnosis Based on Gleason Grading Using Convolutional Neural Network [12.161266795282915]
  We propose a convolutional neural network (CNN)-based automatic classification method for accurate grading of prostate cancer (PCa) using whole slide histopathology images. A data augmentation method named Patch-Based Image Reconstruction (PBIR) was proposed to reduce the high resolution and increase the diversity of WSIs. A distribution correction module was developed to enhance the adaption of pretrained model to the target dataset.
  arXiv  Detail & Related papers  (2020-11-29T06:42:08Z)
• Explaining Clinical Decision Support Systems in Medical Imaging using Cycle-Consistent Activation Maximization [112.2628296775395]
  Clinical decision support using deep neural networks has become a topic of steadily growing interest. clinicians are often hesitant to adopt the technology because its underlying decision-making process is considered to be intransparent and difficult to comprehend. We propose a novel decision explanation scheme based on CycleGAN activation which generates high-quality visualizations of classifier decisions even in smaller data sets.
  arXiv  Detail & Related papers  (2020-10-09T14:39:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.