Enhancing Pathological VLMs with Cross-scale Reasoning

• URL: http://arxiv.org/abs/2606.17412v3
• Date: Wed, 24 Jun 2026 15:20:02 GMT
• Title: Enhancing Pathological VLMs with Cross-scale Reasoning
• Abstract summary: Pathological images are inherently multi-scale, requiring pathologists to integrate evidence from global tissue architecture at low magnification to cellular morphology at higher magnification for accurate diagnosis.<n>We introduce the first cross-scale training and evaluation paradigm that formulates pathology interpretation as multi-magnification reasoning.<n>We present ScaleReasoner-R1, a model trained via reinforcement learning to optimize performance on cross-scale VQA tasks.
• Score: 16.185615513686532
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Pathological images are inherently multi-scale, requiring pathologists to integrate evidence from global tissue architecture at low magnification to cellular morphology at higher magnification for accurate diagnosis. While existing pathological datasets for vision-language models (VLMs) include various scales, they often lack explicit cross-scale reasoning objectives. This limitation prevents VLMs from capturing essential cross-scale representations and learning evidence-based reasoning. To bridge this gap, we introduce the first cross-scale training and evaluation paradigm that formulates pathology interpretation as multi-magnification reasoning. However, creating such a task reveals a critical challenge: multi-image visual question answering (VQA) is prone to text-only shortcuts, which allow models to guess answers using magnification-dependent artifacts rather than visual evidence. To address this, we propose a leakage-aware curation pipeline that combines adversarial text-only screening with constraint-guided question design. Using this pipeline, we construct Scale-VQA, a high-quality benchmark with 4,685 multiple-choice questions grounded in 2,537 pathology images across multiple magnification levels. Finally, we present ScaleReasoner-R1, a model trained via reinforcement learning to optimize performance on cross-scale VQA tasks. ScaleReasoner-R1 achieves state-of-the-art performance on our cross-scale reasoning benchmark and generalizes to SOTA performance on established single-scale benchmarks. Findings suggest that even the limited cross-scale supervision can significantly improve pathological understanding. Code is available at https://github.com/iMVR-PL/ScaleReasoner-R1.

Related papers

• MedFG-VQA: Low-Frequency Memory and Graph Attention for Lightweight Medical VQA [55.12170695219291]
  MedFG-VQA is a lightweight framework that augments DCT-based low-frequency features and employs graph-enhanced cross-attention for effective visual-textual alignment.<n>To address data scarcity, we construct SynMed-VQA, a large-scale synthetic dataset comprising over 2 million question-answer pairs across 9 imaging modalities and 10 major organs.
  arXiv  Detail & Related papers  (2026-08-27T09:16:14Z)
• PathScale-R1: Cross-scale Reasoning for Pathological Image Analysis [22.108085023153482]
  PathScale-VQA is a high-quality cross-scale pathology VQA benchmark with 10,373 multiple-choice questions grounded in 1,368 diagnostic paths across multiple magnification levels.<n>Building on the semantic reasoning set, PathScale-R1 is optimized through Difficulty-driven Reasoning Distillation supervised fine-tuning.
  arXiv  Detail & Related papers  (2026-07-26T18:36:23Z)
• PixDLM: A Dual-Path Multimodal Language Model for UAV Reasoning Segmentation [71.53011159355401]
  Reasoning segmentation has expanded from ground-level scenes to remote-sensing imagery, yet UAV data poses distinct challenges.<n>We define the UAV Reasoning task and organize its semantic requirements into three dimensions: Spatial, Attribute, and Scene-level reasoning.<n>We construct DRSeg, a large-scale benchmark for UAV reasoning segmentation, containing 10k high-resolution aerial images paired with Chain-of-Thought QA supervision.
  arXiv  Detail & Related papers  (2026-04-17T03:48:56Z)
• Context-Aware Asymmetric Ensembling for Interpretable Retinopathy of Prematurity Screening via Active Query and Vascular Attention [1.8420107091891775]
  Retinopathy of Prematurity (ROP) is among the major causes of preventable childhood blindness.<n>Current deep learning models depend heavily on large private datasets and passive multimodal fusion.<n>We propose the Context-Aware Asymmetric Ensemble Model (CAA Ensemble) that simulates clinical reasoning through two specialized streams.
  arXiv  Detail & Related papers  (2026-02-05T02:06:26Z)
• PathReasoner-R1: Instilling Structured Reasoning into Pathology Vision-Language Model via Knowledge-Guided Policy Optimization [6.821738567680833]
  We construct PathReasoner, the first large-scale dataset of whole-slide image (WSI) reasoning.<n>PathReasoner-R1 synergizes supervised fine-tuning with reasoning-oriented reinforcement learning to instill structured chain-of-thought capabilities.<n>Experiments demonstrate that PathReasoner-R1 achieves state-of-the-art performance on both PathReasoner and public benchmarks across various image scales.
  arXiv  Detail & Related papers  (2026-01-29T12:21:16Z)
• More Images, More Problems? A Controlled Analysis of VLM Failure Modes [80.64323947730905]
  Large Vision Language Models (LVLMs) have demonstrated remarkable capabilities, yet their proficiency in understanding and reasoning over multiple images remains largely unexplored.<n>We introduce MIMIC, a new benchmark designed to rigorously evaluate the multi-image capabilities of LVLMs.
  arXiv  Detail & Related papers  (2026-01-12T18:45:13Z)
• Anatomy-R1: Enhancing Anatomy Reasoning in Multimodal Large Language Models via Anatomical Similarity Curriculum and Group Diversity Augmentation [52.7583577508452]
  Multimodal Large Language Models (MLLMs) have achieved impressive progress in natural image reasoning.<n>Their potential in medical imaging remains underexplored, especially in clinical anatomical surgical images.<n>These challenges limit the effectiveness of conventionalSupervised Fine-Tuning strategies.
  arXiv  Detail & Related papers  (2025-12-22T16:06:36Z)
• A Semantically Enhanced Generative Foundation Model Improves Pathological Image Synthesis [82.01597026329158]
  We introduce a Correlation-Regulated Alignment Framework for Tissue Synthesis (CRAFTS) for pathology-specific text-to-image synthesis.<n>CRAFTS incorporates a novel alignment mechanism that suppresses semantic drift to ensure biological accuracy.<n>This model generates diverse pathological images spanning 30 cancer types, with quality rigorously validated by objective metrics and pathologist evaluations.
  arXiv  Detail & Related papers  (2025-12-15T10:22:43Z)
• Navigating Gigapixel Pathology Images with Large Multimodal Models [0.649324006529432]
  General-purpose large multimodal models (LMMs) have generally shown poor or inconclusive performance in medical image interpretation.<n>We introduce Gigapixel Image Agent for Navigating Tissue (GIANT), the first framework that allows LMMs to iteratively navigate whole-slide images like a pathologist.<n>Using MultiPathQA, we show that our simple agentic system substantially outperforms conventional patch- and thumbnail-based baselines.
  arXiv  Detail & Related papers  (2025-11-24T19:33:56Z)
• A Hybrid CNN-VSSM model for Multi-View, Multi-Task Mammography Analysis: Robust Diagnosis with Attention-Based Fusion [5.15423063632115]
  Early and accurate interpretation of screening mammograms is essential for effective breast cancer detection.<n>Existing AI approaches fall short by focusing on single view inputs or single-task outputs.<n>We propose a novel multi-view, multitask hybrid deep learning framework that processes all four standard mammography views.
  arXiv  Detail & Related papers  (2025-07-22T18:52:18Z)
• PathVLM-R1: A Reinforcement Learning-Driven Reasoning Model for Pathology Visual-Language Tasks [15.497221591506625]
  We have proposed PathVLM-R1, a visual language model designed specifically for pathological images.<n>We have based our model on Qwen2.5-VL-7B-Instruct and enhanced its performance for pathological tasks through meticulously designed post-training strategies.
  arXiv  Detail & Related papers  (2025-04-12T15:32:16Z)
• Multi-Modal Prompt Learning on Blind Image Quality Assessment [65.0676908930946]
  Image Quality Assessment (IQA) models benefit significantly from semantic information, which allows them to treat different types of objects distinctly. Traditional methods, hindered by a lack of sufficiently annotated data, have employed the CLIP image-text pretraining model as their backbone to gain semantic awareness. Recent approaches have attempted to address this mismatch using prompt technology, but these solutions have shortcomings. This paper introduces an innovative multi-modal prompt-based methodology for IQA.
  arXiv  Detail & Related papers  (2024-04-23T11:45:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.