Self-Supervised Multi-Modality Learning for Multi-Label Skin Lesion Classification

• URL: http://arxiv.org/abs/2310.18583v2
• Date: Thu, 16 Nov 2023 05:02:34 GMT
• Title: Self-Supervised Multi-Modality Learning for Multi-Label Skin Lesion Classification
• Authors: Hao Wang, Euijoon Ahn, Lei Bi, Jinman Kim
• Abstract summary: We propose a self-supervised learning algorithm for multi-modality skin lesion classification. Our algorithm enables the multi-modality learning by maximizing the similarities between paired dermoscopic and clinical images. Our results show our algorithm achieved better performances than other state-of-the-art SSL counterparts.
• Score: 15.757141597485374
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The clinical diagnosis of skin lesion involves the analysis of dermoscopic and clinical modalities. Dermoscopic images provide a detailed view of the surface structures whereas clinical images offer a complementary macroscopic information. The visual diagnosis of melanoma is also based on seven-point checklist which involves identifying different visual attributes. Recently, supervised learning approaches such as convolutional neural networks (CNNs) have shown great performances using both dermoscopic and clinical modalities (Multi-modality). The seven different visual attributes in the checklist are also used to further improve the the diagnosis. The performances of these approaches, however, are still reliant on the availability of large-scaled labeled data. The acquisition of annotated dataset is an expensive and time-consuming task, more so with annotating multi-attributes. To overcome this limitation, we propose a self-supervised learning (SSL) algorithm for multi-modality skin lesion classification. Our algorithm enables the multi-modality learning by maximizing the similarities between paired dermoscopic and clinical images from different views. In addition, we generate surrogate pseudo-multi-labels that represent seven attributes via clustering analysis. We also propose a label-relation-aware module to refine each pseudo-label embedding and capture the interrelationships between pseudo-multi-labels. We validated the effectiveness of our algorithm using well-benchmarked seven-point skin lesion dataset. Our results show that our algorithm achieved better performances than other state-of-the-art SSL counterparts.

Related papers

• A Novel Perspective for Multi-modal Multi-label Skin Lesion Classification [20.05980794886644]
  This paper introduces the Skin Lesion, utilizing a Multi-modal Multi-label TransFormer-based model (SkinM2Former) SkinM2Former achieves a mean average accuracy of 77.27% and a mean diagnostic accuracy of 77.85% on the public Derm7pt dataset, outperforming state-of-the-art (SOTA) methods.
  arXiv  Detail & Related papers  (2024-09-19T01:31:38Z)
• Pay Less On Clinical Images: Asymmetric Multi-Modal Fusion Method For Efficient Multi-Label Skin Lesion Classification [6.195015783344803]
  Existing multi-modal approaches primarily focus on enhancing multi-label skin lesion classification performance through advanced fusion modules. We introduce a novel asymmetric multi-modal fusion method in this paper for efficient multi-label skin lesion classification.
  arXiv  Detail & Related papers  (2024-07-13T20:46:04Z)
• Multi-task Explainable Skin Lesion Classification [54.76511683427566]
  We propose a few-shot-based approach for skin lesions that generalizes well with few labelled data. The proposed approach comprises a fusion of a segmentation network that acts as an attention module and classification network.
  arXiv  Detail & Related papers  (2023-10-11T05:49:47Z)
• Graph-Ensemble Learning Model for Multi-label Skin Lesion Classification using Dermoscopy and Clinical Images [7.159532626507458]
  This study introduces a Graph Convolution Network (GCN) to exploit prior co-occurrence between each category as a correlation matrix into the deep learning model for the multi-label classification. We propose a Graph-Ensemble Learning Model (GELN) that views the prediction from GCN as complementary information of the predictions from the fusion model.
  arXiv  Detail & Related papers  (2023-07-04T13:19:57Z)
• Malignancy Prediction and Lesion Identification from Clinical Dermatological Images [65.1629311281062]
  We consider machine-learning-based malignancy prediction and lesion identification from clinical dermatological images. We first identify all lesions present in the image regardless of sub-type or likelihood of malignancy, then it estimates their likelihood of malignancy, and through aggregation, it also generates an image-level likelihood of malignancy.
  arXiv  Detail & Related papers  (2021-04-02T20:52:05Z)
• Graph-Based Intercategory and Intermodality Network for Multilabel Classification and Melanoma Diagnosis of Skin Lesions in Dermoscopy and Clinical Images [10.164261744114645]
  Melanoma diagnosis is commonly based on the 7-point visual category checklist (7PC) The 7PC contains intrinsic relationships between categories that can aid classification. Current state-of-the-art methods focus on a single image modality and ignore information from the other. We propose a graph-based intercategory and intermodality network (GIIN) with two modules.
  arXiv  Detail & Related papers  (2021-04-01T02:06:48Z)
• Many-to-One Distribution Learning and K-Nearest Neighbor Smoothing for Thoracic Disease Identification [83.6017225363714]
  deep learning has become the most powerful computer-aided diagnosis technology for improving disease identification performance. For chest X-ray imaging, annotating large-scale data requires professional domain knowledge and is time-consuming. In this paper, we propose many-to-one distribution learning (MODL) and K-nearest neighbor smoothing (KNNS) methods to improve a single model's disease identification performance.
  arXiv  Detail & Related papers  (2021-02-26T02:29:30Z)
• G-MIND: An End-to-End Multimodal Imaging-Genetics Framework for Biomarker Identification and Disease Classification [49.53651166356737]
  We propose a novel deep neural network architecture to integrate imaging and genetics data, as guided by diagnosis, that provides interpretable biomarkers. We have evaluated our model on a population study of schizophrenia that includes two functional MRI (fMRI) paradigms and Single Nucleotide Polymorphism (SNP) data.
  arXiv  Detail & Related papers  (2021-01-27T19:28:04Z)
• Semi-Supervised Active Learning for COVID-19 Lung Ultrasound Multi-symptom Classification [13.878896181984262]
  We propose a novel semi-supervised Two-Stream Active Learning (TSAL) method to model complicated features and reduce labeling costs. On this basis, a multi-symptom multi-label (MSML) classification network is proposed to learn discriminative features of lung symptoms. A novel lung US dataset named COVID19-LUSMS is built, currently containing 71 clinical patients with 6,836 images sampled from 678 videos.
  arXiv  Detail & Related papers  (2020-09-09T10:45:34Z)
• Semi-supervised Medical Image Classification with Relation-driven Self-ensembling Model [71.80319052891817]
  We present a relation-driven semi-supervised framework for medical image classification. It exploits the unlabeled data by encouraging the prediction consistency of given input under perturbations. Our method outperforms many state-of-the-art semi-supervised learning methods on both single-label and multi-label image classification scenarios.
  arXiv  Detail & Related papers  (2020-05-15T06:57:54Z)
• Weakly supervised multiple instance learning histopathological tumor segmentation [51.085268272912415]
  We propose a weakly supervised framework for whole slide imaging segmentation. We exploit a multiple instance learning scheme for training models. The proposed framework has been evaluated on multi-locations and multi-centric public data from The Cancer Genome Atlas and the PatchCamelyon dataset.
  arXiv  Detail & Related papers  (2020-04-10T13:12:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.