How Transferable Are Self-supervised Features in Medical Image Classification Tasks?

• URL: http://arxiv.org/abs/2108.10048v1
• Date: Mon, 23 Aug 2021 10:39:31 GMT
• Title: How Transferable Are Self-supervised Features in Medical Image Classification Tasks?
• Authors: Tuan Truong, Sadegh Mohammadi, Matthias Lenga
• Abstract summary: Transfer learning has become a standard practice to mitigate the lack of labeled data in medical classification tasks. Self-supervised pretrained models yield richer embeddings than their supervised counterpart. Dynamic Visual Meta-Embedding (DVME) is an end-to-end transfer learning approach that fuses pretrained embeddings from multiple models.
• Score: 0.7734726150561086
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Transfer learning has become a standard practice to mitigate the lack of labeled data in medical classification tasks. Whereas finetuning a downstream task using supervised ImageNet pretrained features is straightforward and extensively investigated in many works, there is little study on the usefulness of self-supervised pretraining. In this paper, we assess the transferability of ImageNet self-supervisedpretraining by evaluating the performance of models initialized with pretrained features from three self-supervised techniques (SimCLR, SwAV, and DINO) on selected medical classification tasks. The chosen tasks cover tumor detection in sentinel axillary lymph node images, diabetic retinopathy classification in fundus images, and multiple pathological condition classification in chest X-ray images. We demonstrate that self-supervised pretrained models yield richer embeddings than their supervised counterpart, which benefits downstream tasks in view of both linear evaluation and finetuning. For example, in view of linear evaluation at acritically small subset of the data, we see an improvement up to 14.79% in Kappa score in the diabetic retinopathy classification task, 5.4% in AUC in the tumor classification task, 7.03% AUC in the pneumonia detection, and 9.4% in AUC in the detection of pathological conditions in chest X-ray. In addition, we introduce Dynamic Visual Meta-Embedding (DVME) as an end-to-end transfer learning approach that fuses pretrained embeddings from multiple models. We show that the collective representation obtained by DVME leads to a significant improvement in the performance of selected tasks compared to using a single pretrained model approach and can be generalized to any combination of pretrained models.

Related papers

• Less is More: Selective Reduction of CT Data for Self-Supervised Pre-Training of Deep Learning Models with Contrastive Learning Improves Downstream Classification Performance [7.945551345449388]
  Current findings indicate a strong potential for contrastive pre-training on medical images. We hypothesize that the similarity of medical images hinders the success of contrastive learning in the medical imaging domain. We investigate different strategies based on deep embedding, information theory, and hashing in order to identify and reduce redundancy in medical pre-training datasets.
  arXiv  Detail & Related papers  (2024-10-18T15:08:05Z)
• Computational Pathology at Health System Scale -- Self-Supervised Foundation Models from Three Billion Images [30.618749295623363]
  This project aims to train the largest academic foundation model and benchmark the most prominent self-supervised learning algorithms by pre-training. We collected the largest pathology dataset to date, consisting of over 3 billion images from over 423 thousand microscopy slides. Our results demonstrate that pre-training on pathology data is beneficial for downstream performance compared to pre-training on natural images.
  arXiv  Detail & Related papers  (2023-10-10T21:40:19Z)
• Self-Supervised Pretraining Improves Performance and Inference Efficiency in Multiple Lung Ultrasound Interpretation Tasks [65.23740556896654]
  We investigated whether self-supervised pretraining could produce a neural network feature extractor applicable to multiple classification tasks in lung ultrasound analysis. When fine-tuning on three lung ultrasound tasks, pretrained models resulted in an improvement of the average across-task area under the receiver operating curve (AUC) by 0.032 and 0.061 on local and external test sets respectively.
  arXiv  Detail & Related papers  (2023-09-05T21:36:42Z)
• LVM-Med: Learning Large-Scale Self-Supervised Vision Models for Medical Imaging via Second-order Graph Matching [59.01894976615714]
  We introduce LVM-Med, the first family of deep networks trained on large-scale medical datasets. We have collected approximately 1.3 million medical images from 55 publicly available datasets. LVM-Med empirically outperforms a number of state-of-the-art supervised, self-supervised, and foundation models.
  arXiv  Detail & Related papers  (2023-06-20T22:21:34Z)
• Forward-Forward Contrastive Learning [4.465144120325802]
  We propose Forward Forward Contrastive Learning (FFCL) as a novel pretraining approach for medical image classification. FFCL achieves superior performance (3.69% accuracy over ImageNet pretrained ResNet-18) over existing pretraining models in the pneumonia classification task.
  arXiv  Detail & Related papers  (2023-05-04T15:29:06Z)
• Vision-Language Modelling For Radiological Imaging and Reports In The Low Data Regime [70.04389979779195]
  This paper explores training medical vision-language models (VLMs) where the visual and language inputs are embedded into a common space. We explore several candidate methods to improve low-data performance, including adapting generic pre-trained models to novel image and text domains. Using text-to-image retrieval as a benchmark, we evaluate the performance of these methods with variable sized training datasets of paired chest X-rays and radiological reports.
  arXiv  Detail & Related papers  (2023-03-30T18:20:00Z)
• Significantly improving zero-shot X-ray pathology classification via fine-tuning pre-trained image-text encoders [50.689585476660554]
  We propose a new fine-tuning strategy that includes positive-pair loss relaxation and random sentence sampling. Our approach consistently improves overall zero-shot pathology classification across four chest X-ray datasets and three pre-trained models.
  arXiv  Detail & Related papers  (2022-12-14T06:04:18Z)
• Self-Supervised Learning from Unlabeled Fundus Photographs Improves Segmentation of the Retina [4.815051667870375]
  Fundus photography is the primary method for retinal imaging and essential for diabetic retinopathy prevention. Current segmentation methods are not robust towards the diversity in imaging conditions and pathologies typical for real-world clinical applications. We utilize contrastive self-supervised learning to exploit the large variety of unlabeled fundus images in the publicly available EyePACS dataset.
  arXiv  Detail & Related papers  (2021-08-05T18:02:56Z)
• On the Robustness of Pretraining and Self-Supervision for a Deep Learning-based Analysis of Diabetic Retinopathy [70.71457102672545]
  We compare the impact of different training procedures for diabetic retinopathy grading. We investigate different aspects such as quantitative performance, statistics of the learned feature representations, interpretability and robustness to image distortions. Our results indicate that models from ImageNet pretraining report a significant increase in performance, generalization and robustness to image distortions.
  arXiv  Detail & Related papers  (2021-06-25T08:32:45Z)
• A Multi-Stage Attentive Transfer Learning Framework for Improving COVID-19 Diagnosis [49.3704402041314]
  We propose a multi-stage attentive transfer learning framework for improving COVID-19 diagnosis. Our proposed framework consists of three stages to train accurate diagnosis models through learning knowledge from multiple source tasks and data of different domains. Importantly, we propose a novel self-supervised learning method to learn multi-scale representations for lung CT images.
  arXiv  Detail & Related papers  (2021-01-14T01:39:19Z)
• Big Self-Supervised Models Advance Medical Image Classification [36.23989703428874]
  We study the effectiveness of self-supervised learning as a pretraining strategy for medical image classification. We use a novel Multi-Instance Contrastive Learning (MICLe) method that uses multiple images of the underlying pathology per patient case. We show that big self-supervised models are robust to distribution shift and can learn efficiently with a small number of labeled medical images.
  arXiv  Detail & Related papers  (2021-01-13T17:36:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.