Unsupervised Foundation Model-Agnostic Slide-Level Representation Learning

• URL: http://arxiv.org/abs/2411.13623v1
• Date: Wed, 20 Nov 2024 13:12:43 GMT
• Title: Unsupervised Foundation Model-Agnostic Slide-Level Representation Learning
• Authors: Tim Lenz, Peter Neidlinger, Marta Ligero, Georg Wölflein, Marko van Treeck, Jakob Nikolas Kather,
• Abstract summary: We propose a single modality SSL method in feature space that generates useful slide representations. Our contrastive pretraining strategy, called COBRA, employs multiple FMs and an architecture based on Mamba-2. COBRA exceeds performance of state-of-the-art slide encoders on four different public CPTAC cohorts on average by at least +3.8% AUC.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Representation learning of pathology whole-slide images (WSIs) has primarily relied on weak supervision with Multiple Instance Learning (MIL). This approach leads to slide representations highly tailored to a specific clinical task. Self-supervised learning (SSL) has been successfully applied to train histopathology foundation models (FMs) for patch embedding generation. However, generating patient or slide level embeddings remains challenging. Existing approaches for slide representation learning extend the principles of SSL from patch level learning to entire slides by aligning different augmentations of the slide or by utilizing multimodal data. By integrating tile embeddings from multiple FMs, we propose a new single modality SSL method in feature space that generates useful slide representations. Our contrastive pretraining strategy, called COBRA, employs multiple FMs and an architecture based on Mamba-2. COBRA exceeds performance of state-of-the-art slide encoders on four different public CPTAC cohorts on average by at least +3.8% AUC, despite only being pretrained on 3048 WSIs from TCGA. Additionally, COBRA is readily compatible at inference time with previously unseen feature extractors.

Related papers

• Pathological Prior-Guided Multiple Instance Learning For Mitigating Catastrophic Forgetting in Breast Cancer Whole Slide Image Classification [50.899861205016265]
  We propose a new framework PaGMIL to mitigate catastrophic forgetting in breast cancer WSI classification. Our framework introduces two key components into the common MIL model architecture. We evaluate the continual learning performance of PaGMIL across several public breast cancer datasets.
  arXiv  Detail & Related papers  (2025-03-08T04:51:58Z)
• PMT: Progressive Mean Teacher via Exploring Temporal Consistency for Semi-Supervised Medical Image Segmentation [51.509573838103854]
  We propose a semi-supervised learning framework, termed Progressive Mean Teachers (PMT), for medical image segmentation. Our PMT generates high-fidelity pseudo labels by learning robust and diverse features in the training process. Experimental results on two datasets with different modalities, i.e., CT and MRI, demonstrate that our method outperforms the state-of-the-art medical image segmentation approaches.
  arXiv  Detail & Related papers  (2024-09-08T15:02:25Z)
• SLCA++: Unleash the Power of Sequential Fine-tuning for Continual Learning with Pre-training [68.7896349660824]
  We present an in-depth analysis of the progressive overfitting problem from the lens of Seq FT. Considering that the overly fast representation learning and the biased classification layer constitute this particular problem, we introduce the advanced Slow Learner with Alignment (S++) framework. Our approach involves a Slow Learner to selectively reduce the learning rate of backbone parameters, and a Alignment to align the disjoint classification layers in a post-hoc fashion.
  arXiv  Detail & Related papers  (2024-08-15T17:50:07Z)
• Multistain Pretraining for Slide Representation Learning in Pathology [7.564260323883271]
  Self-supervised learning models can learn universal and transferable representations of whole-slide images. We introduce Madeleine, a multimodal pretraining strategy for slide representation learning. We demonstrate the quality of slide representations learned by Madeleine on various downstream evaluations.
  arXiv  Detail & Related papers  (2024-08-05T22:59:50Z)
• A Multimodal Knowledge-enhanced Whole-slide Pathology Foundation Model [13.96693863133633]
  We curated the largest multimodal dataset consisting of H&E diagnostic whole slide images and their associated pathology reports and RNA-Seq data. We propose a novel whole-slide pretraining paradigm which injects multimodal knowledge at the whole-slide context into the pathology FM. The proposed paradigm revolutionizes the workflow of pretraining for CPath, which enables the pathology FM to acquire the whole-slide context.
  arXiv  Detail & Related papers  (2024-07-22T04:09:27Z)
• Pathology-knowledge Enhanced Multi-instance Prompt Learning for Few-shot Whole Slide Image Classification [19.070685830687285]
  In clinical settings, restricted access to pathology slides is inevitable due to patient privacy concerns and the prevalence of rare or emerging diseases. This paper proposes a multi-instance prompt learning framework enhanced with pathology knowledge. Our method demonstrates superior performance in three challenging clinical tasks, significantly outperforming comparative few-shot methods.
  arXiv  Detail & Related papers  (2024-07-15T15:31:55Z)
• Erasing the Bias: Fine-Tuning Foundation Models for Semi-Supervised Learning [4.137391543972184]
  Semi-supervised learning (SSL) has witnessed remarkable progress, resulting in numerous method variations. In this paper, we present a novel SSL approach named FineSSL that significantly addresses this limitation by adapting pre-trained foundation models. We demonstrate that FineSSL sets a new state of the art for SSL on multiple benchmark datasets, reduces the training cost by over six times, and can seamlessly integrate various fine-tuning and modern SSL algorithms.
  arXiv  Detail & Related papers  (2024-05-20T03:33:12Z)
• Transcriptomics-guided Slide Representation Learning in Computational Pathology [9.192285155829499]
  Self-supervised learning (SSL) has been successful in building patch embeddings of small histology images (e.g., 224x224 pixels) Here, we leverage complementary information from gene expression profiles to guide slide representation learning using multimodal pre-training. Our slide and expression (S+E) pre-training strategy, called Tangle, employs modality-specific encoders, the outputs of which are aligned via contrastive learning.
  arXiv  Detail & Related papers  (2024-05-19T17:17:35Z)
• PRISM: A Multi-Modal Generative Foundation Model for Slide-Level Histopathology [9.556246087301883]
  We present a slide-level foundation model for H&E-stained histopathology, PRISM, that builds on Virchow tile embeddings. PRISM produces slide-level embeddings with the ability to generate clinical reports, resulting in several modes of use. Using text prompts, PRISM achieves zero-shot cancer detection and sub-typing performance approaching that of a supervised aggregator model.
  arXiv  Detail & Related papers  (2024-05-16T16:59:12Z)
• A self-supervised framework for learning whole slide representations [52.774822784847565]
  We present Slide Pre-trained Transformers (SPT) for gigapixel-scale self-supervision of whole slide images. We benchmark SPT visual representations on five diagnostic tasks across three biomedical microscopy datasets.
  arXiv  Detail & Related papers  (2024-02-09T05:05:28Z)
• SkeletonMAE: Graph-based Masked Autoencoder for Skeleton Sequence Pre-training [110.55093254677638]
  We propose an efficient skeleton sequence learning framework, named Skeleton Sequence Learning (SSL) In this paper, we build an asymmetric graph-based encoder-decoder pre-training architecture named SkeletonMAE. Our SSL generalizes well across different datasets and outperforms the state-of-the-art self-supervised skeleton-based action recognition methods.
  arXiv  Detail & Related papers  (2023-07-17T13:33:11Z)
• SLCA: Slow Learner with Classifier Alignment for Continual Learning on a Pre-trained Model [73.80068155830708]
  We present an extensive analysis for continual learning on a pre-trained model (CLPM) We propose a simple but extremely effective approach named Slow Learner with Alignment (SLCA) Across a variety of scenarios, our proposal provides substantial improvements for CLPM.
  arXiv  Detail & Related papers  (2023-03-09T08:57:01Z)
• ConCL: Concept Contrastive Learning for Dense Prediction Pre-training in Pathology Images [47.43840961882509]
  Self-supervised learning is appealing to such annotation-heavy tasks. We first benchmark representative SSL methods for dense prediction tasks in pathology images. We propose concept contrastive learning (ConCL), an SSL framework for dense pre-training.
  arXiv  Detail & Related papers  (2022-07-14T08:38:17Z)
• Self-Supervised Deep Learning to Enhance Breast Cancer Detection on Screening Mammography [2.9082470896148425]
  We investigate strong augmentation based self-supervised learning (SSL) techniques to address this problem. Using breast cancer detection as an example, we first identify a mammogram-specific transformation paradigm. We develop a method to convert a pretrained model from making predictions on uniformly tiled patches to whole images, and an attention-based pooling method that improves the classification performance.
  arXiv  Detail & Related papers  (2022-03-16T03:47:01Z)
• Boosting Few-Shot Learning With Adaptive Margin Loss [109.03665126222619]
  This paper proposes an adaptive margin principle to improve the generalization ability of metric-based meta-learning approaches for few-shot learning problems. Extensive experiments demonstrate that the proposed method can boost the performance of current metric-based meta-learning approaches.
  arXiv  Detail & Related papers  (2020-05-28T07:58:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.