Enhancing Skin Disease Diagnosis: Interpretable Visual Concept Discovery with SAM Empowerment

• URL: http://arxiv.org/abs/2409.09520v1
• Date: Sat, 14 Sep 2024 20:11:25 GMT
• Title: Enhancing Skin Disease Diagnosis: Interpretable Visual Concept Discovery with SAM Empowerment
• Authors: Xin Hu, Janet Wang, Jihun Hamm, Rie R Yotsu, Zhengming Ding,
• Abstract summary: Current AI-assisted skin image diagnosis has achieved dermatologist-level performance in classifying skin cancer. We propose a novel Cross-Attentive Fusion framework for interpretable skin lesion diagnosis.
• Score: 41.398287899966995
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Current AI-assisted skin image diagnosis has achieved dermatologist-level performance in classifying skin cancer, driven by rapid advancements in deep learning architectures. However, unlike traditional vision tasks, skin images in general present unique challenges due to the limited availability of well-annotated datasets, complex variations in conditions, and the necessity for detailed interpretations to ensure patient safety. Previous segmentation methods have sought to reduce image noise and enhance diagnostic performance, but these techniques require fine-grained, pixel-level ground truth masks for training. In contrast, with the rise of foundation models, the Segment Anything Model (SAM) has been introduced to facilitate promptable segmentation, enabling the automation of the segmentation process with simple yet effective prompts. Efforts applying SAM predominantly focus on dermatoscopy images, which present more easily identifiable lesion boundaries than clinical photos taken with smartphones. This limitation constrains the practicality of these approaches to real-world applications. To overcome the challenges posed by noisy clinical photos acquired via non-standardized protocols and to improve diagnostic accessibility, we propose a novel Cross-Attentive Fusion framework for interpretable skin lesion diagnosis. Our method leverages SAM to generate visual concepts for skin diseases using prompts, integrating local visual concepts with global image features to enhance model performance. Extensive evaluation on two skin disease datasets demonstrates our proposed method's effectiveness on lesion diagnosis and interpretability.

Related papers

• SkinGEN: an Explainable Dermatology Diagnosis-to-Generation Framework with Interactive Vision-Language Models [52.90397538472582]
  SkinGEN is a diagnosis-to-generation framework that generates reference demonstrations from diagnosis results provided by VLM. We conduct a user study with 32 participants evaluating both the system performance and explainability. Results demonstrate that SkinGEN significantly improves users' comprehension of VLM predictions and fosters increased trust in the diagnostic process.
  arXiv  Detail & Related papers  (2024-04-23T05:36:33Z)
• Optimizing Skin Lesion Classification via Multimodal Data and Auxiliary Task Integration [54.76511683427566]
  This research introduces a novel multimodal method for classifying skin lesions, integrating smartphone-captured images with essential clinical and demographic information. A distinctive aspect of this method is the integration of an auxiliary task focused on super-resolution image prediction. The experimental evaluations have been conducted using the PAD-UFES20 dataset, applying various deep-learning architectures.
  arXiv  Detail & Related papers  (2024-02-16T05:16:20Z)
• Convolutional Neural Networks Towards Facial Skin Lesions Detection [0.0]
  This study contributes by providing a model that facilitates the detection of blemishes and skin lesions on facial images. The proposed method offers advantages such as simple architecture, speed and suitability for image processing.
  arXiv  Detail & Related papers  (2024-02-13T16:52:10Z)
• Revamping AI Models in Dermatology: Overcoming Critical Challenges for Enhanced Skin Lesion Diagnosis [8.430482797862926]
  We present an All-In-One textbfHierarchical-textbfOut of Distribution-textbfClinical Triage model. For a clinical image, our model generates three outputs: a hierarchical prediction, an alert for out-of-distribution images, and a recommendation for dermoscopy. Our versatile model provides valuable decision support for lesion diagnosis and sets a promising precedent for medical AI applications.
  arXiv  Detail & Related papers  (2023-11-02T06:08:49Z)
• AMLP:Adaptive Masking Lesion Patches for Self-supervised Medical Image Segmentation [67.97926983664676]
  Self-supervised masked image modeling has shown promising results on natural images. However, directly applying such methods to medical images remains challenging. We propose a novel self-supervised medical image segmentation framework, Adaptive Masking Lesion Patches (AMLP)
  arXiv  Detail & Related papers  (2023-09-08T13:18:10Z)
• A Novel Multi-Task Model Imitating Dermatologists for Accurate Differential Diagnosis of Skin Diseases in Clinical Images [27.546559936765863]
  A novel multi-task model, namely DermImitFormer, is proposed to fill this gap by imitating dermatologists' diagnostic procedures and strategies. The model simultaneously predicts body parts and lesion attributes in addition to the disease itself, enhancing diagnosis accuracy and improving diagnosis interpretability.
  arXiv  Detail & Related papers  (2023-07-17T08:05:30Z)
• Malignancy Prediction and Lesion Identification from Clinical Dermatological Images [65.1629311281062]
  We consider machine-learning-based malignancy prediction and lesion identification from clinical dermatological images. We first identify all lesions present in the image regardless of sub-type or likelihood of malignancy, then it estimates their likelihood of malignancy, and through aggregation, it also generates an image-level likelihood of malignancy.
  arXiv  Detail & Related papers  (2021-04-02T20:52:05Z)
• Explaining Clinical Decision Support Systems in Medical Imaging using Cycle-Consistent Activation Maximization [112.2628296775395]
  Clinical decision support using deep neural networks has become a topic of steadily growing interest. clinicians are often hesitant to adopt the technology because its underlying decision-making process is considered to be intransparent and difficult to comprehend. We propose a novel decision explanation scheme based on CycleGAN activation which generates high-quality visualizations of classifier decisions even in smaller data sets.
  arXiv  Detail & Related papers  (2020-10-09T14:39:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.