Transcriptomics-guided Slide Representation Learning in Computational Pathology

• URL: http://arxiv.org/abs/2405.11618v1
• Date: Sun, 19 May 2024 17:17:35 GMT
• Title: Transcriptomics-guided Slide Representation Learning in Computational Pathology
• Authors: Guillaume Jaume, Lukas Oldenburg, Anurag Vaidya, Richard J. Chen, Drew F. K. Williamson, Thomas Peeters, Andrew H. Song, Faisal Mahmood,
• Abstract summary: Self-supervised learning (SSL) has been successful in building patch embeddings of small histology images (e.g., 224x224 pixels) Here, we leverage complementary information from gene expression profiles to guide slide representation learning using multimodal pre-training. Our slide and expression (S+E) pre-training strategy, called Tangle, employs modality-specific encoders, the outputs of which are aligned via contrastive learning.
• Score: 9.192285155829499
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Self-supervised learning (SSL) has been successful in building patch embeddings of small histology images (e.g., 224x224 pixels), but scaling these models to learn slide embeddings from the entirety of giga-pixel whole-slide images (WSIs) remains challenging. Here, we leverage complementary information from gene expression profiles to guide slide representation learning using multimodal pre-training. Expression profiles constitute highly detailed molecular descriptions of a tissue that we hypothesize offer a strong task-agnostic training signal for learning slide embeddings. Our slide and expression (S+E) pre-training strategy, called Tangle, employs modality-specific encoders, the outputs of which are aligned via contrastive learning. Tangle was pre-trained on samples from three different organs: liver (n=6,597 S+E pairs), breast (n=1,020), and lung (n=1,012) from two different species (Homo sapiens and Rattus norvegicus). Across three independent test datasets consisting of 1,265 breast WSIs, 1,946 lung WSIs, and 4,584 liver WSIs, Tangle shows significantly better few-shot performance compared to supervised and SSL baselines. When assessed using prototype-based classification and slide retrieval, Tangle also shows a substantial performance improvement over all baselines. Code available at https://github.com/mahmoodlab/TANGLE.

Related papers

• Unsupervised Foundation Model-Agnostic Slide-Level Representation Learning [0.0]
  We propose a single modality SSL method in feature space that generates useful slide representations. Our contrastive pretraining strategy, called COBRA, employs multiple FMs and an architecture based on Mamba-2. COBRA exceeds performance of state-of-the-art slide encoders on four different public CPTAC cohorts on average by at least +3.8% AUC.
  arXiv  Detail & Related papers  (2024-11-20T13:12:43Z)
• Multistain Pretraining for Slide Representation Learning in Pathology [7.564260323883271]
  Self-supervised learning models can learn universal and transferable representations of whole-slide images. We introduce Madeleine, a multimodal pretraining strategy for slide representation learning. We demonstrate the quality of slide representations learned by Madeleine on various downstream evaluations.
  arXiv  Detail & Related papers  (2024-08-05T22:59:50Z)
• Wound Tissue Segmentation in Diabetic Foot Ulcer Images Using Deep Learning: A Pilot Study [5.397013836968946]
  We have created a DFUTissue dataset for the research community to evaluate wound tissue segmentation algorithms. The dataset contains 110 images with tissues labeled by wound experts and 600 unlabeled images. Due to the limited amount of annotated data, our framework consists of both supervised learning (SL) and semi-supervised learning (SSL) phases.
  arXiv  Detail & Related papers  (2024-06-23T05:01:51Z)
• A self-supervised framework for learning whole slide representations [52.774822784847565]
  We present Slide Pre-trained Transformers (SPT) for gigapixel-scale self-supervision of whole slide images. We benchmark SPT visual representations on five diagnostic tasks across three biomedical microscopy datasets.
  arXiv  Detail & Related papers  (2024-02-09T05:05:28Z)
• SegPrompt: Using Segmentation Map as a Better Prompt to Finetune Deep Models for Kidney Stone Classification [62.403510793388705]
  Deep learning has produced encouraging results for kidney stone classification using endoscope images. The shortage of annotated training data poses a severe problem in improving the performance and generalization ability of the trained model. We propose SegPrompt to alleviate the data shortage problems by exploiting segmentation maps from two aspects.
  arXiv  Detail & Related papers  (2023-03-15T01:30:48Z)
• SLCA: Slow Learner with Classifier Alignment for Continual Learning on a Pre-trained Model [73.80068155830708]
  We present an extensive analysis for continual learning on a pre-trained model (CLPM) We propose a simple but extremely effective approach named Slow Learner with Alignment (SLCA) Across a variety of scenarios, our proposal provides substantial improvements for CLPM.
  arXiv  Detail & Related papers  (2023-03-09T08:57:01Z)
• Leveraging the Third Dimension in Contrastive Learning [88.17394309208925]
  Self-Supervised Learning (SSL) methods operate on unlabeled data to learn robust representations useful for downstream tasks. These augmentations ignore the fact that biological vision takes place in an immersive three-dimensional, temporally contiguous environment. We explore two distinct approaches to incorporating depth signals into the SSL framework.
  arXiv  Detail & Related papers  (2023-01-27T15:45:03Z)
• Self-Supervised PPG Representation Learning Shows High Inter-Subject Variability [3.8036939971290007]
  We propose a Self-Supervised Learning (SSL) method with a pretext task of signal reconstruction to learn an informative generalized PPG representation. Results show that in a very limited label data setting (10 samples per class or less), using SSL is beneficial. SSL may pave the way for the broader use of machine learning models on PPG data in label-scarce regimes.
  arXiv  Detail & Related papers  (2022-12-07T19:02:45Z)
• Giga-SSL: Self-Supervised Learning for Gigapixel Images [0.8029049649310211]
  Whole slide images (WSI) are microscopy images of stained tissue slides routinely prepared for diagnosis and treatment selection in medical practice. The current state-of-the-art (SoTA) approach to classify WSI subdivides them into tiles, encodes them by pre-trained networks and applies Multiple Instance Learning (MIL) to train for specific downstream tasks. Here, we propose a strategy of slide level SSL to leverage the large number of WSI without annotations to infer powerful slide representations.
  arXiv  Detail & Related papers  (2022-12-06T19:09:19Z)
• Generative Zero-Shot Learning for Semantic Segmentation of 3D Point Cloud [79.99653758293277]
  We present the first generative approach for both Zero-Shot Learning (ZSL) and Generalized ZSL (GZSL) on 3D data. We show that it reaches or outperforms the state of the art on ModelNet40 classification for both inductive ZSL and inductive GZSL. Our experiments show that our method outperforms strong baselines, which we additionally propose for this task.
  arXiv  Detail & Related papers  (2021-08-13T13:29:27Z)
• Medical Instrument Segmentation in 3D US by Hybrid Constrained Semi-Supervised Learning [62.13520959168732]
  We propose a semi-supervised learning framework for instrument segmentation in 3D US. To achieve the SSL learning, a Dual-UNet is proposed to segment the instrument. Our proposed method achieves Dice score of about 68.6%-69.1% and the inference time of about 1 sec. per volume.
  arXiv  Detail & Related papers  (2021-07-30T07:59:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.