Meta-Transfer Derm-Diagnosis: Exploring Few-Shot Learning and Transfer Learning for Skin Disease Classification in Long-Tail Distribution

• URL: http://arxiv.org/abs/2404.16814v2
• Date: Wed, 01 Oct 2025 18:09:15 GMT
• Title: Meta-Transfer Derm-Diagnosis: Exploring Few-Shot Learning and Transfer Learning for Skin Disease Classification in Long-Tail Distribution
• Authors: Zeynep Özdemir, Hacer Yalim Keles, Ömer Özgür Tanrıöver,
• Abstract summary: Building accurate models for rare skin diseases remains challenging.<n>We compare three learning strategies: episodic learning, supervised transfer learning, and contrastive self-supervised pretraining.<n>Traditional transfer learning approaches consistently outperform episodic and self-supervised methods as the number of training examples increases.
• Score: 0.9384603486206738
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Building accurate models for rare skin diseases remains challenging due to the lack of sufficient labeled data and the inherently long-tailed distribution of available samples. These issues are further complicated by inconsistencies in how datasets are collected and their varying objectives. To address these challenges, we compare three learning strategies: episodic learning, supervised transfer learning, and contrastive self-supervised pretraining, within a few-shot learning framework. We evaluate five training setups on three benchmark datasets: ISIC2018, Derm7pt, and SD-198. Our findings show that traditional transfer learning approaches, particularly those based on MobileNetV2 and Vision Transformer (ViT) architectures, consistently outperform episodic and self-supervised methods as the number of training examples increases. When combined with batch-level data augmentation techniques such as MixUp, CutMix, and ResizeMix, these models achieve state-of-the-art performance on the SD-198 and Derm7pt datasets, and deliver highly competitive results on ISIC2018. All the source codes related to this work will be made publicly available soon at the provided URL.

Related papers

• Continual Self-Supervised Learning with Masked Autoencoders in Remote Sensing [6.0163252984457145]
  We propose a novel continual self-supervised learning method in the context of masked autoencoders (denoted as CoSMAE)<n> Experimental results show that CoSMAE achieves significant improvements of up to 4.94% over state-of-the-art CL methods applied to MAE.
  arXiv  Detail & Related papers  (2025-06-26T14:28:59Z)
• Style transfer as data augmentation: evaluating unpaired image-to-image translation models in mammography [0.0]
  Deep learning models can learn to detect breast cancer from mammograms. However, challenges with overfitting and poor generalisability prevent their routine use in the clinic. Data augmentation techniques can be used to improve generalisability.
  arXiv  Detail & Related papers  (2025-02-04T16:52:45Z)
• Latent Drifting in Diffusion Models for Counterfactual Medical Image Synthesis [55.959002385347645]
  Latent Drifting enables diffusion models to be conditioned for medical images fitted for the complex task of counterfactual image generation. We evaluate our method on three public longitudinal benchmark datasets of brain MRI and chest X-rays for counterfactual image generation.
  arXiv  Detail & Related papers  (2024-12-30T01:59:34Z)
• Towards Scalable Foundation Models for Digital Dermatology [35.62296620281727]
  We utilize self-supervised learning (SSL) techniques to pre-train models on a dataset of over 240,000 dermatological images. Results show that models pre-trained in this work not only outperform general-purpose models but also approach the performance of models 50 times larger on clinically relevant diagnostic tasks.
  arXiv  Detail & Related papers  (2024-11-08T12:19:20Z)
• Multi-OCT-SelfNet: Integrating Self-Supervised Learning with Multi-Source Data Fusion for Enhanced Multi-Class Retinal Disease Classification [2.5091334993691206]
  Development of a robust deep-learning model for retinal disease diagnosis requires a substantial dataset for training. The capacity to generalize effectively on smaller datasets remains a persistent challenge. We've combined a wide range of data sources to improve performance and generalization to new data.
  arXiv  Detail & Related papers  (2024-09-17T17:22:35Z)
• Few-Shot Airway-Tree Modeling using Data-Driven Sparse Priors [0.0]
  Few-shot learning approaches are cost-effective to transfer pre-trained models using only limited annotated data. We train a data-driven sparsification module to enhance airways efficiently in lung CT scans. We then incorporate these sparse representations in a standard supervised segmentation pipeline as a pretraining step to enhance the performance of the DL models.
  arXiv  Detail & Related papers  (2024-07-05T13:46:11Z)
• Diffusion-Based Neural Network Weights Generation [80.89706112736353]
  D2NWG is a diffusion-based neural network weights generation technique that efficiently produces high-performing weights for transfer learning. Our method extends generative hyper-representation learning to recast the latent diffusion paradigm for neural network weights generation. Our approach is scalable to large architectures such as large language models (LLMs), overcoming the limitations of current parameter generation techniques.
  arXiv  Detail & Related papers  (2024-02-28T08:34:23Z)
• Continual Learning with Pre-Trained Models: A Survey [61.97613090666247]
  Continual Learning aims to overcome the catastrophic forgetting of former knowledge when learning new ones. This paper presents a comprehensive survey of the latest advancements in PTM-based CL.
  arXiv  Detail & Related papers  (2024-01-29T18:27:52Z)
• Cross-head mutual Mean-Teaching for semi-supervised medical image segmentation [6.738522094694818]
  Semi-supervised medical image segmentation (SSMIS) has witnessed substantial advancements by leveraging limited labeled data and abundant unlabeled data. Existing state-of-the-art (SOTA) methods encounter challenges in accurately predicting labels for the unlabeled data. We propose a novel Cross-head mutual mean-teaching Network (CMMT-Net) incorporated strong-weak data augmentation.
  arXiv  Detail & Related papers  (2023-10-08T09:13:04Z)
• The effect of data augmentation and 3D-CNN depth on Alzheimer's Disease detection [51.697248252191265]
  This work summarizes and strictly observes best practices regarding data handling, experimental design, and model evaluation. We focus on Alzheimer's Disease (AD) detection, which serves as a paradigmatic example of challenging problem in healthcare. Within this framework, we train predictive 15 models, considering three different data augmentation strategies and five distinct 3D CNN architectures.
  arXiv  Detail & Related papers  (2023-09-13T10:40:41Z)
• Reinforcement Learning Based Multi-modal Feature Fusion Network for Novel Class Discovery [47.28191501836041]
  In this paper, we employ a Reinforcement Learning framework to simulate the cognitive processes of humans. We also deploy a Member-to-Leader Multi-Agent framework to extract and fuse features from multi-modal information. We demonstrate the performance of our approach in both the 3D and 2D domains by employing the OS-MN40, OS-MN40-Miss, and Cifar10 datasets.
  arXiv  Detail & Related papers  (2023-08-26T07:55:32Z)
• Domain Generalization for Mammographic Image Analysis with Contrastive Learning [62.25104935889111]
  The training of an efficacious deep learning model requires large data with diverse styles and qualities. A novel contrastive learning is developed to equip the deep learning models with better style generalization capability. The proposed method has been evaluated extensively and rigorously with mammograms from various vendor style domains and several public datasets.
  arXiv  Detail & Related papers  (2023-04-20T11:40:21Z)
• Realistic Data Enrichment for Robust Image Segmentation in Histopathology [2.248423960136122]
  We propose a new approach, based on diffusion models, which can enrich an imbalanced dataset with plausible examples from underrepresented groups. Our method can simply expand limited clinical datasets making them suitable to train machine learning pipelines.
  arXiv  Detail & Related papers  (2023-04-19T09:52:50Z)
• Performance of GAN-based augmentation for deep learning COVID-19 image classification [57.1795052451257]
  The biggest challenge in the application of deep learning to the medical domain is the availability of training data. Data augmentation is a typical methodology used in machine learning when confronted with a limited data set. In this work, a StyleGAN2-ADA model of Generative Adversarial Networks is trained on the limited COVID-19 chest X-ray image set.
  arXiv  Detail & Related papers  (2023-04-18T15:39:58Z)
• Diffusion-based Data Augmentation for Skin Disease Classification: Impact Across Original Medical Datasets to Fully Synthetic Images [2.5075774184834803]
  Deep neural networks still rely on large amounts of training data to avoid overfitting. Labeled training data for real-world applications such as healthcare is limited and difficult to access. We build upon the emerging success of text-to-image diffusion probabilistic models in augmenting the training samples of our macroscopic skin disease dataset.
  arXiv  Detail & Related papers  (2023-01-12T04:22:23Z)
• Understanding the Tricks of Deep Learning in Medical Image Segmentation: Challenges and Future Directions [66.40971096248946]
  In this paper, we collect a series of MedISeg tricks for different model implementation phases. We experimentally explore the effectiveness of these tricks on consistent baselines. We also open-sourced a strong MedISeg repository, where each component has the advantage of plug-and-play.
  arXiv  Detail & Related papers  (2022-09-21T12:30:05Z)
• Generative Transfer Learning: Covid-19 Classification with a few Chest X-ray Images [0.0]
  Deep learning models can expedite interpretation and alleviate the work of human experts. Deep Transfer Learning addresses this problem by using a pretrained model in the public domain. We present 1 a simpler generative source model, pretrained on a single but related concept, can perform as effectively as existing larger pretrained models.
  arXiv  Detail & Related papers  (2022-08-10T12:37:52Z)
• 2021 BEETL Competition: Advancing Transfer Learning for Subject Independence & Heterogenous EEG Data Sets [89.84774119537087]
  We design two transfer learning challenges around diagnostics and Brain-Computer-Interfacing (BCI) Task 1 is centred on medical diagnostics, addressing automatic sleep stage annotation across subjects. Task 2 is centred on Brain-Computer Interfacing (BCI), addressing motor imagery decoding across both subjects and data sets.
  arXiv  Detail & Related papers  (2022-02-14T12:12:20Z)
• Team Cogitat at NeurIPS 2021: Benchmarks for EEG Transfer Learning Competition [55.34407717373643]
  Building subject-independent deep learning models for EEG decoding faces the challenge of strong co-shift. Our approach is to explicitly align feature distributions at various layers of the deep learning model. The methodology won first place in the 2021 Benchmarks in EEG Transfer Learning competition, hosted at the NeurIPS conference.
  arXiv  Detail & Related papers  (2022-02-01T11:11:08Z)
• On the Robustness of Pretraining and Self-Supervision for a Deep Learning-based Analysis of Diabetic Retinopathy [70.71457102672545]
  We compare the impact of different training procedures for diabetic retinopathy grading. We investigate different aspects such as quantitative performance, statistics of the learned feature representations, interpretability and robustness to image distortions. Our results indicate that models from ImageNet pretraining report a significant increase in performance, generalization and robustness to image distortions.
  arXiv  Detail & Related papers  (2021-06-25T08:32:45Z)
• Relational Subsets Knowledge Distillation for Long-tailed Retinal Diseases Recognition [65.77962788209103]
  We propose class subset learning by dividing the long-tailed data into multiple class subsets according to prior knowledge. It enforces the model to focus on learning the subset-specific knowledge. The proposed framework proved to be effective for the long-tailed retinal diseases recognition task.
  arXiv  Detail & Related papers  (2021-04-22T13:39:33Z)
• Improving filling level classification with adversarial training [90.01594595780928]
  We investigate the problem of classifying - from a single image - the level of content in a cup or a drinking glass. We use adversarial training in a generic source dataset and then refine the training with a task-specific dataset. We show that transfer learning with adversarial training in the source domain consistently improves the classification accuracy on the test set.
  arXiv  Detail & Related papers  (2021-02-08T08:32:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.