Multi-modal Masked Autoencoders Learn Compositional Histopathological Representations

• URL: http://arxiv.org/abs/2209.01534v1
• Date: Sun, 4 Sep 2022 05:25:31 GMT
• Title: Multi-modal Masked Autoencoders Learn Compositional Histopathological Representations
• Authors: Wisdom Oluchi Ikezogwo, Mehmet Saygin Seyfioglu, Linda Shapiro
• Abstract summary: Masked Autoencoders (MAE) is a recent SSL method suitable for digital pathology. We introduce a multi-modal MAE (MMAE) that leverages the specific compositionality of Hematoxylin & Eosin stained WSIs. Results show that the MMAE architecture outperforms supervised baselines and other state-of-the-art SSL techniques for an eight-class tissue phenotyping task.
• Score: 3.2780506066663655
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Self-supervised learning (SSL) enables learning useful inductive biases through utilizing pretext tasks that require no labels. The unlabeled nature of SSL makes it especially important for whole slide histopathological images (WSIs), where patch-level human annotation is difficult. Masked Autoencoders (MAE) is a recent SSL method suitable for digital pathology as it does not require negative sampling and requires little to no data augmentations. However, the domain shift between natural images and digital pathology images requires further research in designing MAE for patch-level WSIs. In this paper, we investigate several design choices for MAE in histopathology. Furthermore, we introduce a multi-modal MAE (MMAE) that leverages the specific compositionality of Hematoxylin & Eosin (H&E) stained WSIs. We performed our experiments on the public patch-level dataset NCT-CRC-HE-100K. The results show that the MMAE architecture outperforms supervised baselines and other state-of-the-art SSL techniques for an eight-class tissue phenotyping task, utilizing only 100 labeled samples for fine-tuning. Our code is available at https://github.com/wisdomikezogwo/MMAE_Pathology

Related papers

• From Pixels to Histopathology: A Graph-Based Framework for Interpretable Whole Slide Image Analysis [81.19923502845441]
  We develop a graph-based framework that constructs WSI graph representations. We build tissue representations (nodes) that follow biological boundaries rather than arbitrary patches. In our method's final step, we solve the diagnostic task through a graph attention network.
  arXiv  Detail & Related papers  (2025-03-14T20:15:04Z)
• Pathological Prior-Guided Multiple Instance Learning For Mitigating Catastrophic Forgetting in Breast Cancer Whole Slide Image Classification [50.899861205016265]
  We propose a new framework PaGMIL to mitigate catastrophic forgetting in breast cancer WSI classification. Our framework introduces two key components into the common MIL model architecture. We evaluate the continual learning performance of PaGMIL across several public breast cancer datasets.
  arXiv  Detail & Related papers  (2025-03-08T04:51:58Z)
• Multistain Pretraining for Slide Representation Learning in Pathology [7.564260323883271]
  Self-supervised learning models can learn universal and transferable representations of whole-slide images. We introduce Madeleine, a multimodal pretraining strategy for slide representation learning. We demonstrate the quality of slide representations learned by Madeleine on various downstream evaluations.
  arXiv  Detail & Related papers  (2024-08-05T22:59:50Z)
• A self-supervised framework for learning whole slide representations [52.774822784847565]
  We present Slide Pre-trained Transformers (SPT) for gigapixel-scale self-supervision of whole slide images. We benchmark SPT visual representations on five diagnostic tasks across three biomedical microscopy datasets.
  arXiv  Detail & Related papers  (2024-02-09T05:05:28Z)
• Learned representation-guided diffusion models for large-image generation [58.192263311786824]
  We introduce a novel approach that trains diffusion models conditioned on embeddings from self-supervised learning (SSL) Our diffusion models successfully project these features back to high-quality histopathology and remote sensing images. Augmenting real data by generating variations of real images improves downstream accuracy for patch-level and larger, image-scale classification tasks.
  arXiv  Detail & Related papers  (2023-12-12T14:45:45Z)
• GenSelfDiff-HIS: Generative Self-Supervision Using Diffusion for Histopathological Image Segmentation [5.049466204159458]
  Self-supervised learning (SSL) is an alternative paradigm that provides some respite by constructing models utilizing only the unannotated data. In this paper, we propose an SSL approach for segmenting histopathological images via generative diffusion models. Our method is based on the observation that diffusion models effectively solve an image-to-image translation task akin to a segmentation task.
  arXiv  Detail & Related papers  (2023-09-04T09:49:24Z)
• Active Learning Enhances Classification of Histopathology Whole Slide Images with Attention-based Multiple Instance Learning [48.02011627390706]
  We train an attention-based MIL and calculate a confidence metric for every image in the dataset to select the most uncertain WSIs for expert annotation. With a novel attention guiding loss, this leads to an accuracy boost of the trained models with few regions annotated for each class. It may in the future serve as an important contribution to train MIL models in the clinically relevant context of cancer classification in histopathology.
  arXiv  Detail & Related papers  (2023-03-02T15:18:58Z)
• Self Supervised Learning for Few Shot Hyperspectral Image Classification [57.2348804884321]
  We propose to leverage Self Supervised Learning (SSL) for HSI classification. We show that by pre-training an encoder on unlabeled pixels using Barlow-Twins, a state-of-the-art SSL algorithm, we can obtain accurate models with a handful of labels.
  arXiv  Detail & Related papers  (2022-06-24T07:21:53Z)
• Global Contrast Masked Autoencoders Are Powerful Pathological Representation Learners [11.162001837248166]
  We propose a self-supervised learning model, the global contrast-masked autoencoder (GCMAE), which can train the encoder to have the ability to represent local-global features of pathological images. The ability of the GCMAE to learn migratable representations was demonstrated through extensive experiments using a total of three different disease-specific hematoxylin and eosin (HE)-stained pathology datasets.
  arXiv  Detail & Related papers  (2022-05-18T16:28:56Z)
• Self-distillation Augmented Masked Autoencoders for Histopathological Image Classification [11.573165017470867]
  Masked autoencoders (MAE) building self-supervised learning (SSL) from a generative paradigm is probably a more appropriate pre-training. A novel SD-MAE model is proposed to enable a self-distillation augmented MAE. Experiments demonstrate that SD-MAE shows highly competitive performance when compared with other SSL methods.
  arXiv  Detail & Related papers  (2022-03-31T12:09:20Z)
• Label Cleaning Multiple Instance Learning: Refining Coarse Annotations on Single Whole-Slide Images [83.7047542725469]
  Annotating cancerous regions in whole-slide images (WSIs) of pathology samples plays a critical role in clinical diagnosis, biomedical research, and machine learning algorithms development. We present a method, named Label Cleaning Multiple Instance Learning (LC-MIL), to refine coarse annotations on a single WSI without the need of external training data. Our experiments on a heterogeneous WSI set with breast cancer lymph node metastasis, liver cancer, and colorectal cancer samples show that LC-MIL significantly refines the coarse annotations, outperforming the state-of-the-art alternatives, even while learning from a single slide.
  arXiv  Detail & Related papers  (2021-09-22T15:06:06Z)
• DSAL: Deeply Supervised Active Learning from Strong and Weak Labelers for Biomedical Image Segmentation [13.707848142719424]
  We propose a deep active semi-supervised learning framework, DSAL, combining active learning and semi-supervised learning strategies. In DSAL, a new criterion based on deep supervision mechanism is proposed to select informative samples with high uncertainties. We use the proposed criteria to select samples for strong and weak labelers to produce oracle labels and pseudo labels simultaneously at each active learning iteration.
  arXiv  Detail & Related papers  (2021-01-22T11:31:33Z)
• AdarGCN: Adaptive Aggregation GCN for Few-Shot Learning [112.95742995816367]
  We propose a new few-shot fewshot learning setting termed FSFSL. Under FSFSL, both the source and target classes have limited training samples. We also propose a graph convolutional network (GCN)-based label denoising (LDN) method to remove irrelevant images.
  arXiv  Detail & Related papers  (2020-02-28T10:34:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.