Domain Adaptive Skin Lesion Classification via Conformal Ensemble of Vision Transformers

• URL: http://arxiv.org/abs/2505.15997v1
• Date: Wed, 21 May 2025 20:28:43 GMT
• Title: Domain Adaptive Skin Lesion Classification via Conformal Ensemble of Vision Transformers
• Authors: Mehran Zoravar, Shadi Alijani, Homayoun Najjaran,
• Abstract summary: This paper proposes a novel framework termed Conformal Ensemble of Vision Transformers (CE-ViTs)<n>It is designed to enhance image classification performance by prioritizing domain adaptation and model robustness, while accounting for uncertainty.<n>The framework achieves a high coverage rate of 90.38%, representing an improvement of 9.95% compared to the HAM10000 model.
• Score: 3.7305040207339286
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Exploring the trustworthiness of deep learning models is crucial, especially in critical domains such as medical imaging decision support systems. Conformal prediction has emerged as a rigorous means of providing deep learning models with reliable uncertainty estimates and safety guarantees. However, conformal prediction results face challenges due to the backbone model's struggles in domain-shifted scenarios, such as variations in different sources. To aim this challenge, this paper proposes a novel framework termed Conformal Ensemble of Vision Transformers (CE-ViTs) designed to enhance image classification performance by prioritizing domain adaptation and model robustness, while accounting for uncertainty. The proposed method leverages an ensemble of vision transformer models in the backbone, trained on diverse datasets including HAM10000, Dermofit, and Skin Cancer ISIC datasets. This ensemble learning approach, calibrated through the combined mentioned datasets, aims to enhance domain adaptation through conformal learning. Experimental results underscore that the framework achieves a high coverage rate of 90.38\%, representing an improvement of 9.95\% compared to the HAM10000 model. This indicates a strong likelihood that the prediction set includes the true label compared to singular models. Ensemble learning in CE-ViTs significantly improves conformal prediction performance, increasing the average prediction set size for challenging misclassified samples from 1.86 to 3.075.

Related papers

• Conformal Prediction for Zero-Shot Models [20.94974284175104]
  We investigate the capabilities of CLIP models under the split conformal prediction paradigm.<n>We propose Conf-OT, a transfer learning setting that operates transductive over the combined calibration and query sets.
  arXiv  Detail & Related papers  (2025-05-30T15:16:19Z)
• Adaptive Prompt Tuning: Vision Guided Prompt Tuning with Cross-Attention for Fine-Grained Few-Shot Learning [5.242869847419834]
  Few-shot, fine-grained classification in computer vision poses significant challenges due to the need to differentiate subtle class distinctions with limited data.<n>This paper presents a novel method that enhances the Contrastive Language-Image Pre-Training model through adaptive prompt tuning.
  arXiv  Detail & Related papers  (2024-12-19T08:51:01Z)
• Are foundation models for computer vision good conformal predictors? [17.53651859360999]
  We study the behaviour of vision and vision-language foundation models under Conformal Prediction (CP)<n>Our findings reveal that foundation models are well-suited for conformalization procedures, particularly those integrating Vision Transformers.<n>We also show that few-shot adaptation of Vision-Language Models (VLMs) to downstream tasks, whose popularity is surging, enhances conformal scores compared to zero-shot predictions.
  arXiv  Detail & Related papers  (2024-12-08T22:05:38Z)
• Generalized Face Forgery Detection via Adaptive Learning for Pre-trained Vision Transformer [54.32283739486781]
  We present a textbfForgery-aware textbfAdaptive textbfVision textbfTransformer (FA-ViT) under the adaptive learning paradigm. FA-ViT achieves 93.83% and 78.32% AUC scores on Celeb-DF and DFDC datasets in the cross-dataset evaluation.
  arXiv  Detail & Related papers  (2023-09-20T06:51:11Z)
• Consistency Regularization for Generalizable Source-free Domain Adaptation [62.654883736925456]
  Source-free domain adaptation (SFDA) aims to adapt a well-trained source model to an unlabelled target domain without accessing the source dataset. Existing SFDA methods ONLY assess their adapted models on the target training set, neglecting the data from unseen but identically distributed testing sets. We propose a consistency regularization framework to develop a more generalizable SFDA method.
  arXiv  Detail & Related papers  (2023-08-03T07:45:53Z)
• Towards Better Certified Segmentation via Diffusion Models [62.21617614504225]
  segmentation models can be vulnerable to adversarial perturbations, which hinders their use in critical-decision systems like healthcare or autonomous driving. Recently, randomized smoothing has been proposed to certify segmentation predictions by adding Gaussian noise to the input to obtain theoretical guarantees. In this paper, we address the problem of certifying segmentation prediction using a combination of randomized smoothing and diffusion models.
  arXiv  Detail & Related papers  (2023-06-16T16:30:39Z)
• Stacking Ensemble Learning in Deep Domain Adaptation for Ophthalmic Image Classification [61.656149405657246]
  Domain adaptation is effective in image classification tasks where obtaining sufficient label data is challenging. We propose a novel method, named SELDA, for stacking ensemble learning via extending three domain adaptation methods. The experimental results using Age-Related Eye Disease Study (AREDS) benchmark ophthalmic dataset demonstrate the effectiveness of the proposed model.
  arXiv  Detail & Related papers  (2022-09-27T14:19:00Z)
• Transformer Uncertainty Estimation with Hierarchical Stochastic Attention [8.95459272947319]
  We propose a novel way to enable transformers to have the capability of uncertainty estimation. This is achieved by learning a hierarchical self-attention that attends to values and a set of learnable centroids. We empirically evaluate our model on two text classification tasks with both in-domain (ID) and out-of-domain (OOD) datasets.
  arXiv  Detail & Related papers  (2021-12-27T16:43:31Z)
• A Variational Bayesian Approach to Learning Latent Variables for Acoustic Knowledge Transfer [55.20627066525205]
  We propose a variational Bayesian (VB) approach to learning distributions of latent variables in deep neural network (DNN) models. Our proposed VB approach can obtain good improvements on target devices, and consistently outperforms 13 state-of-the-art knowledge transfer algorithms.
  arXiv  Detail & Related papers  (2021-10-16T15:54:01Z)
• Closer Look at the Uncertainty Estimation in Semantic Segmentation under Distributional Shift [2.05617385614792]
  Uncertainty estimation for the task of semantic segmentation is evaluated under a varying level of domain shift. It was shown that simple color transformations already provide a strong baseline. ensemble of models was utilized in the self-training setting to improve the pseudo-labels generation.
  arXiv  Detail & Related papers  (2021-05-31T19:50:43Z)
• From Sound Representation to Model Robustness [82.21746840893658]
  We investigate the impact of different standard environmental sound representations (spectrograms) on the recognition performance and adversarial attack robustness of a victim residual convolutional neural network. Averaged over various experiments on three environmental sound datasets, we found the ResNet-18 model outperforms other deep learning architectures.
  arXiv  Detail & Related papers  (2020-07-27T17:30:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.