PathGLS: Evaluating Pathology Vision-Language Models without Ground Truth through Multi-Dimensional Consistency

• URL: http://arxiv.org/abs/2603.16113v1
• Date: Tue, 17 Mar 2026 04:25:55 GMT
• Title: PathGLS: Evaluating Pathology Vision-Language Models without Ground Truth through Multi-Dimensional Consistency
• Abstract summary: Vision-Language Models (VLMs) offer significant potential in computational pathology by enabling interpretable image analysis, automated reporting, and scalable decision support.<n>To address this gap, we propose PathGLS, a novel reference-free evaluation framework that assesses pathology VLMs across three dimensions.<n> Experiments on Quilt-1M, TCGA, REG2025, PathMMU and TCGA-Sarcoma datasets demonstrate the superiority of PathGLS.
• Score: 22.13541624406203
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-Language Models (VLMs) offer significant potential in computational pathology by enabling interpretable image analysis, automated reporting, and scalable decision support. However, their widespread clinical adoption remains limited due to the absence of reliable, automated evaluation metrics capable of identifying subtle failures such as hallucinations. To address this gap, we propose PathGLS, a novel reference-free evaluation framework that assesses pathology VLMs across three dimensions: Grounding (fine-grained visual-text alignment), Logic (entailment graph consistency using Natural Language Inference), and Stability (output variance under adversarial visual-semantic perturbations). PathGLS supports both patch-level and whole-slide image (WSI)-level analysis, yielding a comprehensive trust score. Experiments on Quilt-1M, TCGA, REG2025, PathMMU and TCGA-Sarcoma datasets demonstrate the superiority of PathGLS. Specifically, on the Quilt-1M dataset, PathGLS reveals a steep sensitivity drop of 40.2% for hallucinated reports compared to only 2.1% for BERTScore. Moreover, validation against expert-defined clinical error hierarchies reveals that PathGLS achieves a strong Spearman's rank correlation of $ρ=0.71$ ($p < 0.0001$), significantly outperforming Large Language Model (LLM)-based approaches (Gemini 3.0 Pro: $ρ=0.39$, $p < 0.0001$). These results establish PathGLS as a robust reference-free metric. By directly quantifying hallucination rates and domain shift robustness, it serves as a reliable criterion for benchmarking VLMs on private clinical datasets and informing safe deployment. Code can be found at: https://github.com/My13ad/PathGLS

Related papers

• NovGauge: A Fine-Grained Benchmark for Diagnosing LLMs' Capability in Paper Novelty Assessment [54.4265627247104]
  Large language models (LLMs) are increasingly used in peer review at major AI conferences.<n>Existing benchmarks assess novelty as a single holistic score.<n>We present NovGauge, a human-anchored benchmark for fine-grained novelty assessment diagnosis.
  arXiv  Detail & Related papers  (2026-09-10T08:34:56Z)
• Synergistic Perception-Reasoning Governance: Grounding Medical MLLMs with Verifiable Anatomical Evidence [45.78409219344429]
  Multimodal large language models (MLLMs) show strong promise for clinical VQA and radiology report generation, yet inference-time hallucinations undermine trustworthy use.<n>We propose a holistic, training-free evidence-injection framework that systematically mitigates hallucinations through dual-side evidence injection.<n>We conduct systematic evaluations on 2 tasks and 5 datasets using textttLLaVA-1.5-7B, texttLLaVA-Med-1.5-7B, texttQwen3-VL-8B/32B, and textttInternVL
  arXiv  Detail & Related papers  (2026-06-30T08:07:25Z)
• Beyond Scalar Scores: Exploring LLM-based Metrics for Clinical Significance Evaluation in Radiology Reports [49.5225801722164]
  Existing metrics obscure this requirement by reducing report quality to a medically ungrounded scalar.<n>We study this boundary using ReEvalMed benchmark as testbed and evaluate metric-level clinical significance.
  arXiv  Detail & Related papers  (2026-06-17T08:10:30Z)
• Latent Performance Profiling of Large Language Models [47.009623327601226]
  We introduce Latent Performance Profiling (LPP) -- a framework that derives task-agnostic diagnostics from hidden activations and output distributions.<n>Unlike static accuracy scores, LPP provides stable, architecture-sensitive signatures across models of similar size.<n>We show that models with similar benchmark scores can exhibit contrasting latent profiles, such as differences in entropy or adaptability.
  arXiv  Detail & Related papers  (2026-05-28T14:41:26Z)
• Beyond Binary Success: A Diagnostic Meta-Evaluation Framework for Fine-Grained Manipulation [98.79811866787263]
  We introduce MetaFine, a diagnostic meta-evaluation framework that disentangles manipulation competency along three axes.<n>We identify the visual encoder's ability to preserve local spatial structure as a key bottleneck for fine-grained precision.<n>By shifting evaluation from ranking to diagnosis, MetaFine turns benchmarking into an actionable compass for repairing the layered capacities underlying genuine physical dexterity.
  arXiv  Detail & Related papers  (2026-05-19T15:25:13Z)
• Geometry-Aware Uncertainty Coresets for Robust Visual In-Context Learning in Histopathology [8.2867621159451]
  We propose GAUC, a training-free coreset selection method operating directly in the pre-trained multimodal embedding space.<n>On CRC-100K and MHIST across multiple open-source VLM architectures, GAUC consistently improves accuracy, calibration, and prompt gradient robustness.
  arXiv  Detail & Related papers  (2026-05-18T13:54:04Z)
• ProtoMedAgent: Multimodal Clinical Interpretability via Privacy-Aware Agentic Workflows [7.27267618572654]
  ProtoMedAgent formalizes multimodal clinical reporting as an iterative, zero-gradient test-time optimization problem.<n>Online generation is strictly constrained by exact set-theoretic differentials and a reflective Scribe-Critic loop.<n>ProtoMedAgent achieves 91.2% Comparison Set Faithfulness where it fundamentally outperforms standard RAG (46.2%)
  arXiv  Detail & Related papers  (2026-05-13T20:57:37Z)
• Enhancing Fine-Grained Spatial Grounding in 3D CT Report Generation via Discriminative Guidance [16.566888073449714]
  Vision--language models (VLMs) for radiology report generation (RRG) can produce long-form chest CT reports from volumetric scans.<n>Existing methods face two key limitations: (i) training supervision is often coarse, aligning a whole CT volume with a full free-text report without explicit alignment for fine-grained attributes or pathology locations.<n>We propose emphDiscrimi Cue-Prompting with Prompt Dropout (DCP-PD), a plug-and-play framework that distills fine-grained cues from free-text reports and uses them to guide report generation.
  arXiv  Detail & Related papers  (2026-04-12T03:25:41Z)
• AgriPath: A Systematic Exploration of Architectural Trade-offs for Crop Disease Classification [45.52399819498234]
  This work presents a systematic empirical comparison of three model paradigms for fine-grained crop disease classification.<n>We introduce AgriPath-LF16, a benchmark containing 111k images spanning 16 crops and 41 diseases.<n>CNNs achieve the highest accuracy on lab imagery but degrade under domain shift.<n> Contrastive VLMs provide a robust and parameter-efficient alternative with competitive cross-domain performance.
  arXiv  Detail & Related papers  (2026-03-08T17:28:01Z)
• Logit-Level Uncertainty Quantification in Vision-Language Models for Histopathology Image Analysis [0.5879782260984691]
  Vision-Language Models (VLMs) with their multimodal capabilities have demonstrated remarkable success in almost all domains.<n>This study proposes a logit-level uncertainty quantification framework for histopathology image analysis using VLMs.
  arXiv  Detail & Related papers  (2026-03-03T21:21:00Z)
• Suppressing Prior-Comparison Hallucinations in Radiology Report Generation via Semantically Decoupled Latent Steering [94.37535002230504]
  We develop a training-free, inference-time control framework termed Semantically Decoupled Latent Steering.<n>Our approach constructs a semantic-free intervention vector via large language model (LLM)-driven semantic decomposition.<n>We show that our approach significantly reduces the probability of historical hallucinations.
  arXiv  Detail & Related papers  (2026-02-27T04:49:01Z)
• PathCo-LatticE: Pathology-Constrained Lattice-Of Experts Framework for Fully-supervised Few-Shot Cardiac MRI Segmentation [0.0]
  Few-shot learning mitigates data scarcity in cardiac MRI segmentation.<n>PathCo-LatticE replaces unlabeled data with pathology-guided synthetic supervision.
  arXiv  Detail & Related papers  (2025-12-10T15:59:43Z)
• Automated Identification of Incidentalomas Requiring Follow-Up: A Multi-Anatomy Evaluation of LLM-Based and Supervised Approaches [5.958100741754613]
  We evaluated large language models (LLMs) against supervised baselines for fine-grained, lesion-level detection of incidentalomas.<n>We introduced a novel inference strategy using lesion-tagged inputs and anatomy-aware prompting to ground model reasoning.<n>The anatomy-informed GPT-OSS-20b model achieved the highest performance, yielding an incidentaloma-positive macro-F1 of 0.79.
  arXiv  Detail & Related papers  (2025-12-05T08:49:57Z)
• MeCaMIL: Causality-Aware Multiple Instance Learning for Fair and Interpretable Whole Slide Image Diagnosis [40.3028468133626]
  Multiple instance learning (MIL) has emerged as the dominant paradigm for whole slide image (WSI) analysis in computational pathology.<n>textbfMeCaMIL, a causality-aware MIL framework, explicitly models demographic confounders through structured causal graphs.<n>MeCaMIL achieves superior fairness -- demographic disparity variance drops by over 65% relative reduction on average across attributes.
  arXiv  Detail & Related papers  (2025-11-14T06:47:21Z)
• TUMLS: Trustful Fully Unsupervised Multi-Level Segmentation for Whole Slide Images of Histology [41.94295877935867]
  We present a trustful fully unsupervised multi-level segmentation methodology (TUMLS) for whole slide images (WSIs)<n>TUMLS adopts an autoencoder (AE) as a feature extractor to identify the different tissue types within low-resolution training data.<n>This solution integrates seamlessly into clinicians, transforming the examination of a whole WSI into a review of concise, interpretable cross-level insights.
  arXiv  Detail & Related papers  (2025-04-17T07:48:05Z)
• Phikon-v2, A large and public feature extractor for biomarker prediction [42.52549987351643]
  We train a vision transformer using DINOv2 and publicly release one iteration of this model for further experimentation, coined Phikon-v2. While trained on publicly available histology slides, Phikon-v2 surpasses our previously released model (Phikon) and performs on par with other histopathology foundation models (FM) trained on proprietary data.
  arXiv  Detail & Related papers  (2024-09-13T20:12:29Z)
• Localizing Anomalies via Multiscale Score Matching Analysis [13.898576482792173]
  This paper introduces Spatial-MSMA, a novel unsupervised method for anomaly localization in brain MRIs. We employ a flexible normalizing flow model conditioned on patch positions and global image features to estimate patch-wise anomaly scores. The method is evaluated on a dataset of 1,650 T1- and T2-weighted brain MRIs from typically developing children.
  arXiv  Detail & Related papers  (2024-06-28T17:57:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.