Parameter-Efficient Fine-Tuning for Medical Image Analysis: The Missed Opportunity

• URL: http://arxiv.org/abs/2305.08252v4
• Date: Mon, 10 Jun 2024 15:11:40 GMT
• Title: Parameter-Efficient Fine-Tuning for Medical Image Analysis: The Missed Opportunity
• Authors: Raman Dutt, Linus Ericsson, Pedro Sanchez, Sotirios A. Tsaftaris, Timothy Hospedales,
• Abstract summary: The application of. Efficient Fine-Tuning (PEFT) in medical image analysis is relatively unexplored. This study fills this gap by evaluating 17 distinct PEFT algorithms on image classification and text-to-image generation tasks. Our findings demonstrate PEFT's effectiveness, particularly in low data regimes common in medical imaging.
• Score: 15.404013190033242
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Foundation models have significantly advanced medical image analysis through the pre-train fine-tune paradigm. Among various fine-tuning algorithms, Parameter-Efficient Fine-Tuning (PEFT) is increasingly utilized for knowledge transfer across diverse tasks, including vision-language and text-to-image generation. However, its application in medical image analysis is relatively unexplored due to the lack of a structured benchmark for evaluating PEFT methods. This study fills this gap by evaluating 17 distinct PEFT algorithms across convolutional and transformer-based networks on image classification and text-to-image generation tasks using six medical datasets of varying size, modality, and complexity. Through a battery of over 700 controlled experiments, our findings demonstrate PEFT's effectiveness, particularly in low data regimes common in medical imaging, with performance gains of up to 22% in discriminative and generative tasks. These recommendations can assist the community in incorporating PEFT into their workflows and facilitate fair comparisons of future PEFT methods, ensuring alignment with advancements in other areas of machine learning and AI.

Related papers

• Probing the Efficacy of Federated Parameter-Efficient Fine-Tuning of Vision Transformers for Medical Image Classification [16.070261684997362]
  Fine-tuning pre-trained models for various downstream tasks is a critical problem in the medical imaging domain. Large size of these models necessitates the use of parameter-efficient fine-tuning (PEFT) to reduce the communication burden in federated learning. In this work, we investigate various federated PEFT strategies for adapting a Vision Transformer (ViT) model for medical image classification.
  arXiv  Detail & Related papers  (2024-07-16T10:28:50Z)
• Understanding differences in applying DETR to natural and medical images [16.200340490559338]
  Transformer-based detectors have shown success in computer vision tasks with natural images. Medical imaging data presents unique challenges such as extremely large image sizes, fewer and smaller regions of interest, and object classes which can be differentiated only through subtle differences. This study evaluates the applicability of these transformer-based design choices when applied to a screening mammography dataset.
  arXiv  Detail & Related papers  (2024-05-27T22:06:42Z)
• EAFP-Med: An Efficient Adaptive Feature Processing Module Based on Prompts for Medical Image Detection [27.783012550610387]
  Cross-domain adaptive medical image detection is challenging due to the differences in lesion representations across various medical imaging technologies. We propose EAFP-Med, an efficient adaptive feature processing module based on prompts for medical image detection. EAFP-Med can efficiently extract lesion features from various medical images based on prompts, enhancing the model's performance.
  arXiv  Detail & Related papers  (2023-11-27T05:10:15Z)
• On Sensitivity and Robustness of Normalization Schemes to Input Distribution Shifts in Automatic MR Image Diagnosis [58.634791552376235]
  Deep Learning (DL) models have achieved state-of-the-art performance in diagnosing multiple diseases using reconstructed images as input. DL models are sensitive to varying artifacts as it leads to changes in the input data distribution between the training and testing phases. We propose to use other normalization techniques, such as Group Normalization and Layer Normalization, to inject robustness into model performance against varying image artifacts.
  arXiv  Detail & Related papers  (2023-06-23T03:09:03Z)
• Understanding the Tricks of Deep Learning in Medical Image Segmentation: Challenges and Future Directions [66.40971096248946]
  In this paper, we collect a series of MedISeg tricks for different model implementation phases. We experimentally explore the effectiveness of these tricks on consistent baselines. We also open-sourced a strong MedISeg repository, where each component has the advantage of plug-and-play.
  arXiv  Detail & Related papers  (2022-09-21T12:30:05Z)
• Development of an algorithm for medical image segmentation of bone tissue in interaction with metallic implants [58.720142291102135]
  This study develops an algorithm for calculating bone growth in contact with metallic implants. Bone and implant tissue were manually segmented in the training data set. In terms of network accuracy, the model reached around 98%.
  arXiv  Detail & Related papers  (2022-04-22T08:17:20Z)
• Harmonizing Pathological and Normal Pixels for Pseudo-healthy Synthesis [68.5287824124996]
  We present a new type of discriminator, the segmentor, to accurately locate the lesions and improve the visual quality of pseudo-healthy images. We apply the generated images into medical image enhancement and utilize the enhanced results to cope with the low contrast problem. Comprehensive experiments on the T2 modality of BraTS demonstrate that the proposed method substantially outperforms the state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-03-29T08:41:17Z)
• Generative Adversarial U-Net for Domain-free Medical Image Augmentation [49.72048151146307]
  The shortage of annotated medical images is one of the biggest challenges in the field of medical image computing. In this paper, we develop a novel generative method named generative adversarial U-Net. Our newly designed model is domain-free and generalizable to various medical images.
  arXiv  Detail & Related papers  (2021-01-12T23:02:26Z)
• An encoder-decoder-based method for COVID-19 lung infection segmentation [3.561478746634639]
  This paper proposes a multi-task deep-learning-based method for lung infection segmentation using CT-scan images. The proposed method can segment lung infections with a high degree performance even with shortage of data and labeled images.
  arXiv  Detail & Related papers  (2020-07-02T04:02:03Z)
• Unified Representation Learning for Efficient Medical Image Analysis [0.623075162128532]
  We propose a multi-task training approach for medical image analysis using a unified modality-specific feature representation (UMS-Rep) Our results demonstrate that the proposed approach reduces the overall demand for computational resources and improves target task generalization and performance.
  arXiv  Detail & Related papers  (2020-06-19T16:52:16Z)
• Pathological Retinal Region Segmentation From OCT Images Using Geometric Relation Based Augmentation [84.7571086566595]
  We propose improvements over previous GAN-based medical image synthesis methods by jointly encoding the intrinsic relationship of geometry and shape. The proposed method outperforms state-of-the-art segmentation methods on the public RETOUCH dataset having images captured from different acquisition procedures.
  arXiv  Detail & Related papers  (2020-03-31T11:50:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.