Leveraging Knowledge Distillation for Lightweight Skin Cancer Classification: Balancing Accuracy and Computational Efficiency

• URL: http://arxiv.org/abs/2406.17051v2
• Date: Fri, 28 Jun 2024 05:50:11 GMT
• Title: Leveraging Knowledge Distillation for Lightweight Skin Cancer Classification: Balancing Accuracy and Computational Efficiency
• Authors: Niful Islam, Khan Md Hasib, Fahmida Akter Joti, Asif Karim, Sami Azam,
• Abstract summary: Skin cancer is a major concern to public health, accounting for one-third of the reported cancers. We present a knowledge distillation based approach for creating a lightweight yet high-performing classifier. With its high accuracy and compact size, our model appears to be a potential choice for accurate skin cancer classification, particularly in resource-constrained settings.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Skin cancer is a major concern to public health, accounting for one-third of the reported cancers. If not detected early, the cancer has the potential for severe consequences. Recognizing the critical need for effective skin cancer classification, we address the limitations of existing models, which are often too large to deploy in areas with limited computational resources. In response, we present a knowledge distillation based approach for creating a lightweight yet high-performing classifier. The proposed solution involves fusing three models, namely ResNet152V2, ConvNeXtBase, and ViT Base, to create an effective teacher model. The teacher model is then employed to guide a lightweight student model of size 2.03 MB. This student model is further compressed to 469.77 KB using 16-bit quantization, enabling smooth incorporation into edge devices. With six-stage image preprocessing, data augmentation, and a rigorous ablation study, the model achieves an impressive accuracy of 98.75% on the HAM10000 dataset and 98.94% on the Kaggle dataset in classifying benign and malignant skin cancers. With its high accuracy and compact size, our model appears to be a potential choice for accurate skin cancer classification, particularly in resource-constrained settings.

Related papers

• Cancer-Net PCa-Seg: Benchmarking Deep Learning Models for Prostate Cancer Segmentation Using Synthetic Correlated Diffusion Imaging [65.83291923029985]
  Prostate cancer (PCa) is the most prevalent cancer among men in the United States, accounting for nearly 300,000 cases, 29% of all diagnoses and 35,000 total deaths in 2024. Traditional screening methods such as prostate-specific antigen (PSA) testing and magnetic resonance imaging (MRI) have been pivotal in diagnosis, but have faced limitations in specificity and generalizability. We employ several state-of-the-art deep learning models, including U-Net, SegResNet, Swin UNETR, Attention U-Net, and LightM-UNet, to segment PCa lesions from a 200 CDI$
  arXiv  Detail & Related papers  (2025-01-15T22:23:41Z)
• Breast Tumor Classification Using EfficientNet Deep Learning Model [5.062500255359342]
  We introduce an intensive data augmentation pipeline and cost-sensitive learning, improving representation and ensuring that the model does not overly favor majority classes. Our results underscore significant improvements in the binary classification performance, achieving an exceptional recall increase for benign cases.
  arXiv  Detail & Related papers  (2024-11-26T20:38:33Z)
• Cancer-Net SCa-Synth: An Open Access Synthetically Generated 2D Skin Lesion Dataset for Skin Cancer Classification [65.83291923029985]
  In the United States, skin cancer ranks as the most commonly diagnosed cancer, presenting a significant public health issue. Recent advancements in dataset curation and deep learning have shown promise in quick and accurate detection of skin cancer. Cancer-Net SCa- Synth is an open access synthetically generated 2D skin lesion dataset for skin cancer classification.
  arXiv  Detail & Related papers  (2024-11-08T02:04:21Z)
• Brain Tumor Classification on MRI in Light of Molecular Markers [61.77272414423481]
  Co-deletion of the 1p/19q gene is associated with clinical outcomes in low-grade gliomas. This study aims to utilize a specially MRI-based convolutional neural network for brain cancer detection.
  arXiv  Detail & Related papers  (2024-09-29T07:04:26Z)
• Skin Cancer Images Classification using Transfer Learning Techniques [0.0]
  Early diagnosis of skin cancer at a benign stage is critical to reducing cancer mortality. Previous studies have addressed the problem of skin cancer diagnosis using various deep learning and transfer learning models. In this work, we applied five different pre-trained transfer learning approaches for binary classification of skin cancer detection at benign and malignant stages.
  arXiv  Detail & Related papers  (2024-06-18T15:48:20Z)
• Boosting Medical Image-based Cancer Detection via Text-guided Supervision from Reports [68.39938936308023]
  We propose a novel text-guided learning method to achieve highly accurate cancer detection results. Our approach can leverage clinical knowledge by large-scale pre-trained VLM to enhance generalization ability.
  arXiv  Detail & Related papers  (2024-05-23T07:03:38Z)
• Diagnosis of Skin Cancer Using VGG16 and VGG19 Based Transfer Learning Models [0.6827423171182154]
  Deep convolution neural networks (CNN) have shown an excellent potential for data and image classification. In this article, we inspect skin lesion classification problem using CNN techniques. We present that prominent classification accuracy of lesion detection can be obtained by proper designing and applying of transfer learning framework.
  arXiv  Detail & Related papers  (2024-04-01T15:06:20Z)
• Explainable Knowledge Distillation for On-device Chest X-Ray Classification [7.319145625411966]
  We propose a knowledge distillation (KD) strategy to create the compact deep learning model for the real-time multi-label CXR image classification. Our results on three benchmark CXR datasets show that our KD strategy provides the improved performance on the compact student model.
  arXiv  Detail & Related papers  (2023-05-10T15:25:05Z)
• Multi-class Skin Cancer Classification Architecture Based on Deep Convolutional Neural Network [2.4469484645516837]
  This paper proposes a computer-based deep learning approach that will accurately identify different kinds of skin lesions. Deep learning approaches can detect skin cancer very accurately since the models learn each pixel of an image. Some deep learning models have limitations, leading the model to a false-positive result.
  arXiv  Detail & Related papers  (2023-03-13T23:16:18Z)
• New pyramidal hybrid textural and deep features based automatic skin cancer classification model: Ensemble DarkNet and textural feature extractor [1.4502611532302039]
  This research interests to overcome automatic skin cancer detection problem. An automatic multilevel textural and deep features-based model is presented. The chosen top 1000 features are classified using the 10-fold cross-validation technique.
  arXiv  Detail & Related papers  (2022-03-28T20:53:09Z)
• SSD-KD: A Self-supervised Diverse Knowledge Distillation Method for Lightweight Skin Lesion Classification Using Dermoscopic Images [62.60956024215873]
  Skin cancer is one of the most common types of malignancy, affecting a large population and causing a heavy economic burden worldwide. Most studies in skin cancer detection keep pursuing high prediction accuracies without considering the limitation of computing resources on portable devices. This study specifically proposes a novel method, termed SSD-KD, that unifies diverse knowledge into a generic KD framework for skin diseases classification.
  arXiv  Detail & Related papers  (2022-03-22T06:54:29Z)
• EMT-NET: Efficient multitask network for computer-aided diagnosis of breast cancer [58.720142291102135]
  We propose an efficient and light-weighted learning architecture to classify and segment breast tumors simultaneously. We incorporate a segmentation task into a tumor classification network, which makes the backbone network learn representations focused on tumor regions. The accuracy, sensitivity, and specificity of tumor classification is 88.6%, 94.1%, and 85.3%, respectively.
  arXiv  Detail & Related papers  (2022-01-13T05:24:40Z)
• Many-to-One Distribution Learning and K-Nearest Neighbor Smoothing for Thoracic Disease Identification [83.6017225363714]
  deep learning has become the most powerful computer-aided diagnosis technology for improving disease identification performance. For chest X-ray imaging, annotating large-scale data requires professional domain knowledge and is time-consuming. In this paper, we propose many-to-one distribution learning (MODL) and K-nearest neighbor smoothing (KNNS) methods to improve a single model's disease identification performance.
  arXiv  Detail & Related papers  (2021-02-26T02:29:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.