Multimodal system for skin cancer detection

• URL: http://arxiv.org/abs/2601.14822v1
• Date: Wed, 21 Jan 2026 09:50:13 GMT
• Title: Multimodal system for skin cancer detection
• Authors: Volodymyr Sydorskyi, Igor Krashenyi, Oleksii Yakubenko,
• Abstract summary: This study introduces a multi-modal melanoma detection system using conventional photo images, making it more accessible and versatile.<n>It employs a multi-modal neural network combining image and metadata processing and supports a two-step model for cases with or without metadata.<n>A three-stage pipeline further refines predictions by boosting algorithms and enhancing performance.
• Score: 0.764671395172401
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Melanoma detection is vital for early diagnosis and effective treatment. While deep learning models on dermoscopic images have shown promise, they require specialized equipment, limiting their use in broader clinical settings. This study introduces a multi-modal melanoma detection system using conventional photo images, making it more accessible and versatile. Our system integrates image data with tabular metadata, such as patient demographics and lesion characteristics, to improve detection accuracy. It employs a multi-modal neural network combining image and metadata processing and supports a two-step model for cases with or without metadata. A three-stage pipeline further refines predictions by boosting algorithms and enhancing performance. To address the challenges of a highly imbalanced dataset, specific techniques were implemented to ensure robust training. An ablation study evaluated recent vision architectures, boosting algorithms, and loss functions, achieving a peak Partial ROC AUC of 0.18068 (0.2 maximum) and top-15 retrieval sensitivity of 0.78371. Results demonstrate that integrating photo images with metadata in a structured, multi-stage pipeline yields significant performance improvements. This system advances melanoma detection by providing a scalable, equipment-independent solution suitable for diverse healthcare environments, bridging the gap between specialized and general clinical practices.

Related papers

• Lightweight Relational Embedding in Task-Interpolated Few-Shot Networks for Enhanced Gastrointestinal Disease Classification [0.0]
  Colon cancer detection is crucial for increasing patient survival rates.<n> colonoscopy is dependent on obtaining adequate and high-quality endoscopic images.<n>Few-Shot Learning architecture enables our model to rapidly adapt to unseen fine-grained endoscopic image patterns.<n>Our model demonstrated superior performance, achieving an accuracy of 90.1%, precision of 0.845, recall of 0.942, and an F1 score of 0.891.
  arXiv  Detail & Related papers  (2025-05-30T16:54:51Z)
• Evaluation of Vision Transformers for Multimodal Image Classification: A Case Study on Brain, Lung, and Kidney Tumors [0.0]
  The work evaluates the performance of Vision Transformers architectures, including Swin Transformer and MaxViT, in several datasets of MRI and CT scans.<n>The results revealed that the Swin Transformer provided high accuracy, achieving up to 99% on average for individual datasets and 99.4% accuracy for the combined dataset.
  arXiv  Detail & Related papers  (2025-02-08T10:35:51Z)
• A Unified Model for Compressed Sensing MRI Across Undersampling Patterns [69.19631302047569]
  We propose a unified MRI reconstruction model robust to various measurement undersampling patterns and image resolutions.<n>Our model improves SSIM by 11% and PSNR by 4 dB over a state-of-the-art CNN (End-to-End VarNet) with 600$times$ faster inference than diffusion methods.
  arXiv  Detail & Related papers  (2024-10-05T20:03:57Z)
• Multiscale Latent Diffusion Model for Enhanced Feature Extraction from Medical Images [5.395912799904941]
  variations in CT scanner models and acquisition protocols introduce significant variability in the extracted radiomic features.<n> LTDiff++ is a multiscale latent diffusion model designed to enhance feature extraction in medical imaging.
  arXiv  Detail & Related papers  (2024-10-05T02:13:57Z)
• Enhancing Skin Disease Diagnosis: Interpretable Visual Concept Discovery with SAM [41.398287899966995]
  Current AI-assisted skin image diagnosis has achieved dermatologist-level performance in classifying skin cancer.<n>We propose a novel Cross-Attentive Fusion framework for interpretable skin lesion diagnosis.
  arXiv  Detail & Related papers  (2024-09-14T20:11:25Z)
• Adapting Visual-Language Models for Generalizable Anomaly Detection in Medical Images [68.42215385041114]
  This paper introduces a novel lightweight multi-level adaptation and comparison framework to repurpose the CLIP model for medical anomaly detection. Our approach integrates multiple residual adapters into the pre-trained visual encoder, enabling a stepwise enhancement of visual features across different levels. Our experiments on medical anomaly detection benchmarks demonstrate that our method significantly surpasses current state-of-the-art models.
  arXiv  Detail & Related papers  (2024-03-19T09:28:19Z)
• Rapid hyperspectral photothermal mid-infrared spectroscopic imaging from sparse data for gynecologic cancer tissue subtyping [3.550171634694342]
  Mid-infrared (MIR) hyperspectral photothermal imaging is a label-free, biochemically quantitative technology. This work presents a novel approach to MIR photothermal imaging that enhances its speed by an order of magnitude.
  arXiv  Detail & Related papers  (2024-02-28T00:57:35Z)
• Beyond Images: An Integrative Multi-modal Approach to Chest X-Ray Report Generation [47.250147322130545]
  Image-to-text radiology report generation aims to automatically produce radiology reports that describe the findings in medical images. Most existing methods focus solely on the image data, disregarding the other patient information accessible to radiologists. We present a novel multi-modal deep neural network framework for generating chest X-rays reports by integrating structured patient data, such as vital signs and symptoms, alongside unstructured clinical notes.
  arXiv  Detail & Related papers  (2023-11-18T14:37:53Z)
• Negligible effect of brain MRI data preprocessing for tumor segmentation [36.89606202543839]
  We conduct experiments on three publicly available datasets and evaluate the effect of different preprocessing steps in deep neural networks. Our results demonstrate that most popular standardization steps add no value to the network performance. We suggest that image intensity normalization approaches do not contribute to model accuracy because of the reduction of signal variance with image standardization.
  arXiv  Detail & Related papers  (2022-04-11T17:29:36Z)
• EMT-NET: Efficient multitask network for computer-aided diagnosis of breast cancer [58.720142291102135]
  We propose an efficient and light-weighted learning architecture to classify and segment breast tumors simultaneously. We incorporate a segmentation task into a tumor classification network, which makes the backbone network learn representations focused on tumor regions. The accuracy, sensitivity, and specificity of tumor classification is 88.6%, 94.1%, and 85.3%, respectively.
  arXiv  Detail & Related papers  (2022-01-13T05:24:40Z)
• Self-supervised Learning from 100 Million Medical Images [13.958840691105992]
  We propose a method for self-supervised learning of rich image features based on contrastive learning and online feature clustering. We leverage large training datasets of over 100,000,000 medical images of various modalities, including radiography, computed tomography (CT), magnetic resonance (MR) imaging and ultrasonography. We highlight a number of advantages of this strategy on challenging image assessment problems in radiography, CT and MR.
  arXiv  Detail & Related papers  (2022-01-04T18:27:04Z)
• Many-to-One Distribution Learning and K-Nearest Neighbor Smoothing for Thoracic Disease Identification [83.6017225363714]
  deep learning has become the most powerful computer-aided diagnosis technology for improving disease identification performance. For chest X-ray imaging, annotating large-scale data requires professional domain knowledge and is time-consuming. In this paper, we propose many-to-one distribution learning (MODL) and K-nearest neighbor smoothing (KNNS) methods to improve a single model's disease identification performance.
  arXiv  Detail & Related papers  (2021-02-26T02:29:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.