Improved skin lesion recognition by a Self-Supervised Curricular Deep Learning approach

• URL: http://arxiv.org/abs/2112.12086v1
• Date: Wed, 22 Dec 2021 17:45:47 GMT
• Title: Improved skin lesion recognition by a Self-Supervised Curricular Deep Learning approach
• Authors: Kirill Sirotkin (1), Marcos Escudero Vi\~nolo (1), Pablo Carballeira (1), Juan Carlos SanMiguel (1) ((1) Universidad Aut\'onoma de Madrid, Escuela Polit\'ecnica Superior, Spain)
• Abstract summary: State-of-the-art deep learning approaches for skin lesion recognition often require pretraining on larger and more varied datasets. ImageNet is often used as the pretraining dataset, but its transferring potential is hindered by the domain gap between the source dataset and the target dermatoscopic scenario. In this work, we introduce a novel pretraining approach that sequentially trains a series of Self-Supervised Learning pretext tasks.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: State-of-the-art deep learning approaches for skin lesion recognition often require pretraining on larger and more varied datasets, to overcome the generalization limitations derived from the reduced size of the skin lesion imaging datasets. ImageNet is often used as the pretraining dataset, but its transferring potential is hindered by the domain gap between the source dataset and the target dermatoscopic scenario. In this work, we introduce a novel pretraining approach that sequentially trains a series of Self-Supervised Learning pretext tasks and only requires the unlabeled skin lesion imaging data. We present a simple methodology to establish an ordering that defines a pretext task curriculum. For the multi-class skin lesion classification problem, and ISIC-2019 dataset, we provide experimental evidence showing that: i) a model pretrained by a curriculum of pretext tasks outperforms models pretrained by individual pretext tasks, and ii) a model pretrained by the optimal pretext task curriculum outperforms a model pretrained on ImageNet. We demonstrate that this performance gain is related to the fact that the curriculum of pretext tasks better focuses the attention of the final model on the skin lesion. Beyond performance improvement, this strategy allows for a large reduction in the training time with respect to ImageNet pretraining, which is especially advantageous for network architectures tailored for a specific problem.

Related papers

• Enhancing pretraining efficiency for medical image segmentation via transferability metrics [0.0]
  In medical image segmentation tasks, the scarcity of labeled training data poses a significant challenge. We introduce a novel transferability metric, based on contrastive learning, that measures how robustly a pretrained model is able to represent the target data.
  arXiv  Detail & Related papers  (2024-10-24T12:11:52Z)
• Self-Supervised Pre-Training with Contrastive and Masked Autoencoder Methods for Dealing with Small Datasets in Deep Learning for Medical Imaging [8.34398674359296]
  Deep learning in medical imaging has the potential to minimize the risk of diagnostic errors, reduce radiologist workload, and accelerate diagnosis. Training such deep learning models requires large and accurate datasets, with annotations for all training samples. To address this challenge, deep learning models can be pre-trained on large image datasets without annotations using methods from the field of self-supervised learning.
  arXiv  Detail & Related papers  (2023-08-12T11:31:01Z)
• Self-Distillation for Further Pre-training of Transformers [83.84227016847096]
  We propose self-distillation as a regularization for a further pre-training stage. We empirically validate the efficacy of self-distillation on a variety of benchmark datasets for image and text classification tasks.
  arXiv  Detail & Related papers  (2022-09-30T02:25:12Z)
• Self-Supervised-RCNN for Medical Image Segmentation with Limited Data Annotation [0.16490701092527607]
  We propose an alternative deep learning training strategy based on self-supervised pretraining on unlabeled MRI scans. Our pretraining approach first, randomly applies different distortions to random areas of unlabeled images and then predicts the type of distortions and loss of information. The effectiveness of the proposed method for segmentation tasks in different pre-training and fine-tuning scenarios is evaluated.
  arXiv  Detail & Related papers  (2022-07-17T13:28:52Z)
• Intelligent Masking: Deep Q-Learning for Context Encoding in Medical Image Analysis [48.02011627390706]
  We develop a novel self-supervised approach that occludes targeted regions to improve the pre-training procedure. We show that training the agent against the prediction model can significantly improve the semantic features extracted for downstream classification tasks.
  arXiv  Detail & Related papers  (2022-03-25T19:05:06Z)
• On Efficient Transformer and Image Pre-training for Low-level Vision [74.22436001426517]
  Pre-training has marked numerous state of the arts in high-level computer vision. We present an in-depth study of image pre-training. We find pre-training plays strikingly different roles in low-level tasks.
  arXiv  Detail & Related papers  (2021-12-19T15:50:48Z)
• On the Robustness of Pretraining and Self-Supervision for a Deep Learning-based Analysis of Diabetic Retinopathy [70.71457102672545]
  We compare the impact of different training procedures for diabetic retinopathy grading. We investigate different aspects such as quantitative performance, statistics of the learned feature representations, interpretability and robustness to image distortions. Our results indicate that models from ImageNet pretraining report a significant increase in performance, generalization and robustness to image distortions.
  arXiv  Detail & Related papers  (2021-06-25T08:32:45Z)
• Self-Supervised Pretraining Improves Self-Supervised Pretraining [83.1423204498361]
  Self-supervised pretraining requires expensive and lengthy computation, large amounts of data, and is sensitive to data augmentation. This paper explores Hierarchical PreTraining (HPT), which decreases convergence time and improves accuracy by initializing the pretraining process with an existing pretrained model. We show HPT converges up to 80x faster, improves accuracy across tasks, and improves the robustness of the self-supervised pretraining process to changes in the image augmentation policy or amount of pretraining data.
  arXiv  Detail & Related papers  (2021-03-23T17:37:51Z)
• Adversarially-Trained Deep Nets Transfer Better: Illustration on Image Classification [53.735029033681435]
  Transfer learning is a powerful methodology for adapting pre-trained deep neural networks on image recognition tasks to new domains. In this work, we demonstrate that adversarially-trained models transfer better than non-adversarially-trained models.
  arXiv  Detail & Related papers  (2020-07-11T22:48:42Z)
• Multi-task pre-training of deep neural networks for digital pathology [8.74883469030132]
  We first assemble and transform many digital pathology datasets into a pool of 22 classification tasks and almost 900k images. We show that our models used as feature extractors either improve significantly over ImageNet pre-trained models or provide comparable performance.
  arXiv  Detail & Related papers  (2020-05-05T08:50:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.