Skin Lesion Phenotyping via Nested Multi-modal Contrastive Learning

• URL: http://arxiv.org/abs/2505.23709v1
• Date: Thu, 29 May 2025 17:42:13 GMT
• Title: Skin Lesion Phenotyping via Nested Multi-modal Contrastive Learning
• Authors: Dionysis Christopoulos, Sotiris Spanos, Eirini Baltzi, Valsamis Ntouskos, Konstantinos Karantzalos,
• Abstract summary: Melanoma detection and skin lesion classification based solely on images, pose significant challenges.<n>We introduce SLIMP (Skin Lesion Image-Metadata Pre-training) for learning rich representations of skin lesions.
• Score: 2.0721818920754584
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce SLIMP (Skin Lesion Image-Metadata Pre-training) for learning rich representations of skin lesions through a novel nested contrastive learning approach that captures complex relationships between images and metadata. Melanoma detection and skin lesion classification based solely on images, pose significant challenges due to large variations in imaging conditions (lighting, color, resolution, distance, etc.) and lack of clinical and phenotypical context. Clinicians typically follow a holistic approach for assessing the risk level of the patient and for deciding which lesions may be malignant and need to be excised, by considering the patient's medical history as well as the appearance of other lesions of the patient. Inspired by this, SLIMP combines the appearance and the metadata of individual skin lesions with patient-level metadata relating to their medical record and other clinically relevant information. By fully exploiting all available data modalities throughout the learning process, the proposed pre-training strategy improves performance compared to other pre-training strategies on downstream skin lesions classification tasks highlighting the learned representations quality.

Related papers

• An analysis of data variation and bias in image-based dermatological datasets for machine learning classification [2.039829968340841]
  In clinical dermatology, classification models can detect malignant lesions on patients' skin using only RGB images as input.<n>Most learning-based methods employ data acquired from dermoscopic datasets on training, which are large and validated by a gold standard.<n>This work aims to evaluate the gap between dermoscopic and clinical samples and understand how the dataset variations impact training.
  arXiv  Detail & Related papers  (2025-01-15T17:18:46Z)
• FairSkin: Fair Diffusion for Skin Disease Image Generation [54.29840149709033]
  Diffusion Model (DM) has become a leading method in generating synthetic medical images, but it suffers from a critical twofold bias. We propose FairSkin, a novel DM framework that mitigates these biases through a three-level resampling mechanism. Our approach significantly improves the diversity and quality of generated images, contributing to more equitable skin disease detection in clinical settings.
  arXiv  Detail & Related papers  (2024-10-29T21:37:03Z)
• Enhancing Skin Disease Diagnosis: Interpretable Visual Concept Discovery with SAM [41.398287899966995]
  Current AI-assisted skin image diagnosis has achieved dermatologist-level performance in classifying skin cancer.<n>We propose a novel Cross-Attentive Fusion framework for interpretable skin lesion diagnosis.
  arXiv  Detail & Related papers  (2024-09-14T20:11:25Z)
• Optimizing Skin Lesion Classification via Multimodal Data and Auxiliary Task Integration [54.76511683427566]
  This research introduces a novel multimodal method for classifying skin lesions, integrating smartphone-captured images with essential clinical and demographic information. A distinctive aspect of this method is the integration of an auxiliary task focused on super-resolution image prediction. The experimental evaluations have been conducted using the PAD-UFES20 dataset, applying various deep-learning architectures.
  arXiv  Detail & Related papers  (2024-02-16T05:16:20Z)
• Multi-task Explainable Skin Lesion Classification [54.76511683427566]
  We propose a few-shot-based approach for skin lesions that generalizes well with few labelled data. The proposed approach comprises a fusion of a segmentation network that acts as an attention module and classification network.
  arXiv  Detail & Related papers  (2023-10-11T05:49:47Z)
• A Transformer-based representation-learning model with unified processing of multimodal input for clinical diagnostics [63.106382317917344]
  We report a Transformer-based representation-learning model as a clinical diagnostic aid that processes multimodal input in a unified manner. The unified model outperformed an image-only model and non-unified multimodal diagnosis models in the identification of pulmonary diseases.
  arXiv  Detail & Related papers  (2023-06-01T16:23:47Z)
• Benchmarking Heterogeneous Treatment Effect Models through the Lens of Interpretability [82.29775890542967]
  Estimating personalized effects of treatments is a complex, yet pervasive problem. Recent developments in the machine learning literature on heterogeneous treatment effect estimation gave rise to many sophisticated, but opaque, tools. We use post-hoc feature importance methods to identify features that influence the model's predictions.
  arXiv  Detail & Related papers  (2022-06-16T17:59:05Z)
• A Survey on Deep Learning for Skin Lesion Segmentation [32.523358996420846]
  Skin cancer is a major public health problem that could benefit from computer-aided diagnosis to reduce the burden of this common disease. Skin lesion segmentation from images is an important step toward achieving this goal. The presence of natural and artificial artifacts (e.g., hair and air bubbles), intrinsic factors (e.g., lesion shape and contrast), and variations in image acquisition conditions make skin lesion segmentation a challenging task.
  arXiv  Detail & Related papers  (2022-06-01T09:43:10Z)
• Malignancy Prediction and Lesion Identification from Clinical Dermatological Images [65.1629311281062]
  We consider machine-learning-based malignancy prediction and lesion identification from clinical dermatological images. We first identify all lesions present in the image regardless of sub-type or likelihood of malignancy, then it estimates their likelihood of malignancy, and through aggregation, it also generates an image-level likelihood of malignancy.
  arXiv  Detail & Related papers  (2021-04-02T20:52:05Z)
• A Patient-Centric Dataset of Images and Metadata for Identifying Melanomas Using Clinical Context [39.10946113351587]
  The 2020 SIIM-ISIC Melanoma Classification challenge dataset was constructed to address the discrepancy between prior challenges and clinical practice. The dataset represents 2,056 patients from three continents with an average of 16 lesions per patient.
  arXiv  Detail & Related papers  (2020-08-07T20:22:23Z)
• Melanoma Detection using Adversarial Training and Deep Transfer Learning [6.22964000148682]
  We propose a two-stage framework for automatic classification of skin lesion images. In the first stage, we leverage the inter-class variation of the data distribution for the task of conditional image synthesis. In the second stage, we train a deep convolutional neural network for skin lesion classification.
  arXiv  Detail & Related papers  (2020-04-14T22:46:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.