Validation of Whole-Slide Foundation Models for Image Retrieval in TCGA Data

• URL: http://arxiv.org/abs/2605.00902v1
• Date: Tue, 28 Apr 2026 19:19:53 GMT
• Title: Validation of Whole-Slide Foundation Models for Image Retrieval in TCGA Data
• Abstract summary: We benchmarked ten pipelines on 9,387 diagnostic slides spanning 17 organs and 60 diagnoses from The Cancer Genome Atlas.<n>Methods included four pre-trained slide foundation models, a supervised attention-based multiple instance learning (ABMIL) aggregator on patch embeddings, and patch-level retrieval.
• Score: 1.834937230572996
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Foundation models are reshaping computational histopathology, yet their value for whole-slide image retrieval relative to strong patch-based and supervised aggregation baselines remains unclear. We benchmarked ten pipelines on 9,387 diagnostic slides spanning 17 organs and 60 diagnoses from The Cancer Genome Atlas (TCGA) using patient-level leave-one-patient-out evaluation. Methods included four pre-trained slide foundation models, a supervised attention-based multiple instance learning (ABMIL) aggregator on patch embeddings, and patch-level retrieval across five sampling densities. Performance varied more across organs and diagnoses than across architectures. Although the slide foundation model TITAN achieved the strongest overall results, its advantage was modest; ABMIL and patch-based methods reached comparable Top-1 and Top-3 accuracy, with no model consistently dominant. Morphologically distinctive entities approached ceiling performance, while rare, heterogeneous, and closely related subtypes remained challenging. Misclassifications aligned with organs exhibiting known inter-observer variability, suggesting an intrinsic ceiling for morphology-only retrieval. Performance was driven primarily by patch-level feature representations, with limited benefit from slide-level aggregation, indicating aggregation may be unnecessary in many settings. These findings argue against a universally optimal architecture and instead support organ-resolved benchmarking, diagnosis-aware or ensemble strategies, stronger feature representations, and multimodal retrieval frameworks. Notably, even the best model achieved only $\approx 68\% \pm 21\%$ retrieval accuracy on TCGA, and some subtypes showed $0\%$ accuracy across all methods, highlighting fundamental limitations of morphology-based representations and the need for substantial progress before reliable clinical deployment.

Related papers

• Bridging Vision Foundation Model Priors with CLIP for Spatial-aware Few-shot Anomaly Detection in Medical Images [48.646448077284276]
  We propose Spatial-FAD, a framework for effective few-shot medical anomaly detection.<n>It improves lesion localization by combining VFM spatial priors with CLIP semantics.<n>Our method achieves an average improvement of over 11.4% in Dice score and 1.8% in AUC.
  arXiv  Detail & Related papers  (2026-09-11T05:21:01Z)
• Understanding Synergistic Interactions among Pathology Foundation Models via Adaptive Fusion [49.550545038402184]
  Pathology foundation models (PFMs) provide strong tile-level representations via self-supervised pre-training on large-scale pathology images.<n>We present AdaFusion, a lightweight adaptive fusion framework that integrates complementary signals from multiple frozen PFMs.<n>AdaFusion consistently outperforms individual PFMs and other fusion baselines, while providing interpretable tissue visualisation.
  arXiv  Detail & Related papers  (2026-08-02T16:44:45Z)
• In-Context Time Series Classification with Random Convolutional Features [38.40148298084127]
  Time series classification is central to domains like medical signal analysis, industrial monitoring, and sensor-based activity recognition.<n>We propose MASHT, a pipeline that marries MultiRocket and Hydra features with the power of in-context tabular foundation models.<n>Our approach completely bypasses task-specific model training, requiring only feature extraction and direct inference.
  arXiv  Detail & Related papers  (2026-07-21T16:04:35Z)
• PulmoSight-XAI: An Explainable Multi-View Attention Ensemble with Gradient Boosting Meta-Learning for Multi-Label Chest X-Ray Classification [1.2744523252873352]
  We propose an explainable hierarchical multi-view ensemble framework for the robust classification of 14 thoracic pathologies.<n>The framework employs view-specific training by independently modeling frontal and lateral radiographs.<n>The framework achieves state-of-the-art macro-average AUROC scores of 0.9319 for frontal and 0.9154 for lateral radiographs.
  arXiv  Detail & Related papers  (2026-07-05T19:50:53Z)
• Semantic Segmentation for Histopathology using Learned Regularization based on Global Proportions [36.19647813139097]
  We introduce Variational from Label Proportions (VSLP), a two-stage framework that infers dense segmentations from global proportions.<n>Unlike end-to-end networks, our variational method can visualize the fidelity-regularization energy, resulting in more interpretable segmentation.<n>We validate our approach on two public datasets, achieving superior performance over existing supervised and unsupervised methods.
  arXiv  Detail & Related papers  (2026-04-27T11:42:56Z)
• GOLDMARK: Governed Outcome-Linked Diagnostic Model Assessment Reference Kit [7.891604152731754]
  Computational biomarkers (CBs) are histo-derived patterns extracted from hematoxylin-eosin (H&E) whole-slide images (WSIs) to predict therapeutic response or prognosis.<n>We introduce GOLDMARK, a standardized benchmarking framework built on a curated TCGA cohort with clinically actionable OncoKB level 1-3 biomarker labels.
  arXiv  Detail & Related papers  (2026-03-21T15:09:06Z)
• Benchmarking Computational Pathology Foundation Models For Semantic Segmentation [0.0]
  Concatenating features from CONCH, PathDino and CellViT outperformed individual models across all the datasets by 7.95%.<n>We show that the vision language foundation model, CONCH performed the best across datasets when compared to vision-only foundation models.
  arXiv  Detail & Related papers  (2026-02-21T08:00:46Z)
• LapFM: A Laparoscopic Segmentation Foundation Model via Hierarchical Concept Evolving Pre-training [39.51756689308245]
  We present LapFM, a foundation model designed to evolve robust segmentation capabilities from massive unlabeled surgical images.<n>We propose a Confidence-driven Evolving Labeling that iteratively generates and filters pseudo-labels based on hierarchical consistency.<n>This process yields LapBench-114K, a large-scale benchmark comprising 114K image-mask pairs.
  arXiv  Detail & Related papers  (2025-12-09T10:09:31Z)
• Foundation Models in Dermatopathology: Skin Tissue Classification [0.05397680436511065]
  This study evaluates the performance of two foundation models, UNI and Virchow2, as feature extractors for classifying whole-slide images.<n> Patch-level embeddings were aggregated into slide-level features using a mean-aggregation strategy.<n>Results demonstrate that patch-level features extracted using Virchow2 outperformed those extracted via UNI across most slide-level classifiers.
  arXiv  Detail & Related papers  (2025-10-24T17:21:43Z)
• AHDMIL: Asymmetric Hierarchical Distillation Multi-Instance Learning for Fast and Accurate Whole-Slide Image Classification [51.525891360380285]
  AHDMIL is an Asymmetric Hierarchical Distillation Multi-Instance Learning framework.<n>It eliminates irrelevant patches through a two-step training process.<n>It consistently outperforms previous state-of-the-art methods in both classification performance and inference speed.
  arXiv  Detail & Related papers  (2025-08-07T07:47:16Z)
• FundusGAN: A Hierarchical Feature-Aware Generative Framework for High-Fidelity Fundus Image Generation [35.46876389599076]
  FundusGAN is a novel hierarchical feature-aware generative framework specifically designed for high-fidelity fundus image synthesis.<n>We show that FundusGAN consistently outperforms state-of-the-art methods across multiple metrics.
  arXiv  Detail & Related papers  (2025-03-22T18:08:07Z)
• A deep learning framework for efficient pathology image analysis [0.0]
  We introduce Eagle, a framework that emulates pathologists by selectively analyzing informative regions.<n>It processes a slide in 2.27 seconds, reducing computational time by more than 99% compared to existing models.<n>It provides robust and interpretable outputs, supporting rapid slide searches, integration into multi-omics pipelines and emerging clinical foundation models.
  arXiv  Detail & Related papers  (2025-02-18T16:45:01Z)
• Towards A Generalizable Pathology Foundation Model via Unified Knowledge Distillation [41.25398139658467]
  Current foundation models have only been evaluated on a limited type and number of tasks, leaving their generalization ability and overall performance unclear.<n>We establish a benchmark to evaluate the performance of off-the-shelf foundation models across six distinct clinical task types.<n>We propose a unified knowledge distillation framework consisting of both expert and self-knowledge distillation, where the former allows the model to learn from the knowledge of multiple expert models.
  arXiv  Detail & Related papers  (2024-07-26T01:12:54Z)
• Benchmarking Embedding Aggregation Methods in Computational Pathology: A Clinical Data Perspective [32.93871326428446]
  Recent advances in artificial intelligence (AI) are revolutionizing medical imaging and computational pathology.<n>A constant challenge in the analysis of digital Whole Slide Images (WSIs) is the problem of aggregating tens of thousands of tile-level image embeddings to a slide-level representation.<n>This study conducts a benchmarking analysis of ten slide-level aggregation techniques across nine clinically relevant tasks.
  arXiv  Detail & Related papers  (2024-07-10T17:00:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.