Vision-Language Models as Zero-Annotation Oracles in Histopathology

• URL: http://arxiv.org/abs/2606.16658v1
• Date: Mon, 15 Jun 2026 12:50:19 GMT
• Title: Vision-Language Models as Zero-Annotation Oracles in Histopathology
• Abstract summary: Foreground segmentation is the critical first step of every computational pathology pipeline.<n>We propose a coarse-to-fine approach that recasts foreground segmentation as a visual perception task.<n>Our framework provides a principled, scalable solution to a persistent infrastructure bottleneck in digital pathology.
• Score: 10.853300035340615
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Foreground segmentation is the critical first step of every computational pathology pipeline, yet existing methods rely on hand-tuned heuristics or supervised models that overfit to narrow stain and scanner distributions, failing silently on specialised stains such as Jones silver or Elastica van Gieson. We propose a coarse-to-fine approach that recasts foreground segmentation as a visual perception task and leverages general-purpose vision-language models (VLMs) as zero-annotation oracles. Our key insight is that tissue-versus-background discrimination is a natural-image recognition problem, not a histopathological one, so VLMs trained on internet-scale corpora generalise where domain-specific models cannot. We introduce Leica-75, a benchmark of 75 renal transplant whole-slide images spanning three stain families. On Leica-75, our method achieves the highest segmentation quality on out-of-distribution stains (Dice 0.858 +/- 0.027 on Jones, 0.853 +/- 0.041 on EVG) with 7x lower cross-stain variance than the best supervised baseline, while remaining competitive on in-distribution H&E. Few-shot prompting with automatically curated exemplars (Auto-context) rescues hard cases on Stress-32 (n=32), a curated stress-test subset (Dice 0.470 to 0.819 for the 2B model). VLM-based annotation review matches human expert consensus (kappa=0.989 for blur detection; mean precision/recall grading accuracy 0.708 vs. human 0.646 for segmentation mask review). The resulting pseudo-labels are used to distil lightweight student models that are as performant as the teacher model while running for a fraction of the cost. Our framework provides a principled, scalable solution to a persistent infrastructure bottleneck in digital pathology.

Related papers

• EMFE: A lightweight, explainable machine learning framework for malaria cell classification [0.7437884690206271]
  Automated malaria diagnosis from stained blood-smear microscopy is dominated by deep convolutional neural networks.<n>We present EMFE, a five-feature framework for classifying single red-blood-cell images as parasitized or uninfected.
  arXiv  Detail & Related papers  (2026-08-25T16:38:04Z)
• MalariAI: A Label-Resilient Decoupled Framework for Universal Cell Segmentation and Explainable Stage Classification in Dense Malaria Blood Smears [0.0]
  Malaria diagnosis from blood smear microscopy is a critical challenge in global health AI.<n>Three compounding failure modes prevent reliable clinical deployment of existing deep learning systems.<n>We present MalariAI, a two-stage decoupled framework that addresses all three failure modes in a unified pipeline.
  arXiv  Detail & Related papers  (2026-07-01T03:27:04Z)
• From Pixels to Explanations: Interpretable Diabetic Retinopathy Grading with CNN-Transformer Ensembles, Visual Explainability and Vision-Language Models [7.782282985072339]
  This study presents a methodology that combines strong discriminative models with multimodal explanations.<n>For interpretability, we produced Grad-CAM++ visual attribution maps and short textual rationales.<n>For explanation quality, Grad-CAM++ offered plausible but coarse localization, and VLM rationales were generally grade-consistent.
  arXiv  Detail & Related papers  (2026-04-25T00:21:11Z)
• Beyond Anatomy: Explainable ASD Classification from rs-fMRI via Functional Parcellation and Graph Attention Networks [6.923757075165361]
  Anatomical brain parcellations dominate rs-fMRI-based Autism Spectrum Disorder (ASD) classification.<n>We present a graph-based deep learning framework comparing anatomical (AAL) and functionally-derived (MSDL) parcellation strategies on the ABIDE I dataset.
  arXiv  Detail & Related papers  (2026-03-03T02:05:20Z)
• Predicting Diabetic Retinopathy Using a Two-Level Ensemble Model [0.6445605125467574]
  Diabetic retinopathy is a leading cause of blindness in working-age adults.<n>Image-based AI tools have shown limitations in early-stage detection.<n>We propose a non-image-based, two-level ensemble model for DR prediction using routine laboratory test results.
  arXiv  Detail & Related papers  (2025-10-01T16:19:57Z)
• A Novel Attention-Augmented Wavelet YOLO System for Real-time Brain Vessel Segmentation on Transcranial Color-coded Doppler [49.03919553747297]
  We propose an AI-powered, real-time CoW auto-segmentation system capable of efficiently capturing cerebral arteries.<n>No prior studies have explored AI-driven cerebrovascular segmentation using Transcranial Color-coded Doppler (TCCD)<n>The proposed AAW-YOLO demonstrated strong performance in segmenting both ipsilateral and contralateral CoW vessels.
  arXiv  Detail & Related papers  (2025-08-19T14:41:22Z)
• GRASP-PsONet: Gradient-based Removal of Spurious Patterns for PsOriasis Severity Classification [0.0]
  We propose a framework to automatically flag problematic training images that introduce spurious correlations.<n>Removing 8.2% of flagged images improves model AUC-ROC by 5% (85% to 90%) on a held out test set.<n>When applied to a subset of training data rated by two dermatologists, the method identifies over 90% of cases with inter-rater disagreement.
  arXiv  Detail & Related papers  (2025-06-27T03:42:09Z)
• KaLDeX: Kalman Filter based Linear Deformable Cross Attention for Retina Vessel Segmentation [46.57880203321858]
  We propose a novel network (KaLDeX) for vascular segmentation leveraging a Kalman filter based linear deformable cross attention (LDCA) module. Our approach is based on two key components: Kalman filter (KF) based linear deformable convolution (LD) and cross-attention (CA) modules. The proposed method is evaluated on retinal fundus image datasets (DRIVE, CHASE_BD1, and STARE) as well as the 3mm and 6mm of the OCTA-500 dataset.
  arXiv  Detail & Related papers  (2024-10-28T16:00:42Z)
• Uncertainty-inspired Open Set Learning for Retinal Anomaly Identification [71.06194656633447]
  We establish an uncertainty-inspired open-set (UIOS) model, which was trained with fundus images of 9 retinal conditions. Our UIOS model with thresholding strategy achieved an F1 score of 99.55%, 97.01% and 91.91% for the internal testing set. UIOS correctly predicted high uncertainty scores, which would prompt the need for a manual check in the datasets of non-target categories retinal diseases, low-quality fundus images, and non-fundus images.
  arXiv  Detail & Related papers  (2023-04-08T10:47:41Z)
• An Ensemble Method to Automatically Grade Diabetic Retinopathy with Optical Coherence Tomography Angiography Images [4.640835690336653]
  We propose an ensemble method to automatically grade Diabetic retinopathy (DR) images available from Diabetic Retinopathy Analysis Challenge (DRAC) 2022. First, we adopt the state-of-the-art classification networks, and train them to grade UW- OCTA images with different splits of the available dataset. Ultimately, we obtain 25 models, of which, the top 16 models are selected and ensembled to generate the final predictions.
  arXiv  Detail & Related papers  (2022-12-12T22:06:47Z)
• Supervised Contrastive Learning to Classify Paranasal Anomalies in the Maxillary Sinus [43.850343556811275]
  Using deep learning techniques, anomalies in the paranasal sinus system can be detected automatically in MRI images. Existing deep learning methods in paranasal anomaly classification have been used to diagnose at most one anomaly. We propose a novel learning paradigm that combines contrastive loss and cross-entropy loss.
  arXiv  Detail & Related papers  (2022-09-05T12:31:28Z)
• Self-supervised contrastive learning of echocardiogram videos enables label-efficient cardiac disease diagnosis [48.64462717254158]
  We developed a self-supervised contrastive learning approach, EchoCLR, to catered to echocardiogram videos. When fine-tuned on small portions of labeled data, EchoCLR pretraining significantly improved classification performance for left ventricular hypertrophy (LVH) and aortic stenosis (AS) EchoCLR is unique in its ability to learn representations of medical videos and demonstrates that SSL can enable label-efficient disease classification from small, labeled datasets.
  arXiv  Detail & Related papers  (2022-07-23T19:17:26Z)
• (Certified!!) Adversarial Robustness for Free! [116.6052628829344]
  We certify 71% accuracy on ImageNet under adversarial perturbations constrained to be within a 2-norm of 0.5. We obtain these results using only pretrained diffusion models and image classifiers, without requiring any fine tuning or retraining of model parameters.
  arXiv  Detail & Related papers  (2022-06-21T17:27:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.