Parameter-Efficient Fine-Tuning of Large Pretrained Models for Instance Segmentation Tasks

• URL: http://arxiv.org/abs/2606.01947v1
• Date: Mon, 01 Jun 2026 09:09:32 GMT
• Title: Parameter-Efficient Fine-Tuning of Large Pretrained Models for Instance Segmentation Tasks
• Abstract summary: This study investigates the effectiveness of PEFT methods, specifically adapters and Low-Rank Adaptation (LoRA)<n>Using 2-3 adapters per transformer block offers an optimal balance of performance and efficiency.<n>LoRA exhibits strong parameter efficiency when applied to deformable attention, and in certain cases surpasses adapter configurations.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Research and applications in artificial intelligence have recently shifted with the rise of large pretrained models, which deliver state-of-the-art results across numerous tasks. However, the substantial increase in parameters introduces a need for parameter-efficient training strategies. Despite significant advancements, limited research has explored parameter-efficient fine-tuning (PEFT) methods in the context of transformer-based models for instance segmentation. Addressing this gap, this study investigates the effectiveness of PEFT methods, specifically adapters and Low-Rank Adaptation (LoRA), applied to two models across four benchmark datasets. Integrating sequentially arranged adapter modules and applying LoRA to deformable attention--explored here for the first time--achieves competitive performance while fine-tuning only about 1-6% of model parameters, a marked improvement over the 40-55% required in traditional fine-tuning. Key findings indicate that using 2-3 adapters per transformer block offers an optimal balance of performance and efficiency. Furthermore, LoRA, exhibits strong parameter efficiency when applied to deformable attention, and in certain cases surpasses adapter configurations. These results show that the impact of PEFT techniques varies based on dataset complexity and model architecture, underscoring the importance of context-specific tuning. Overall, this work demonstrates the potential of PEFT to enable scalable, customizable, and computationally efficient transfer learning for instance segmentation tasks.

Related papers

• TLoRA+: A Low-Rank Parameter-Efficient Fine-Tuning Method for Large Language Models [4.092357804635167]
  We propose a novel PEFT method that incorporates the TLoRA+ into the weight matrices of pre-trained models.<n>The proposed approach not only preserves the efficiency of low-rank adaptation but also further enhances performance without significantly increasing computational cost.
  arXiv  Detail & Related papers  (2026-04-15T00:30:57Z)
• Efficient Adversarial Training via Criticality-Aware Fine-Tuning [59.84340722595335]
  Criticality-Aware Adversarial Training (CAAT) is a novel method that adaptively allocates resources to the most robustness-critical parameters.<n>CAAT incurs only a 4.3% decrease in adversarial robustness while tuning approximately 6% of its parameters.<n>Experiments on three widely used adversarial learning datasets demonstrate that CAAT outperforms state-of-the-art lightweight AT methods with fewer trainable parameters.
  arXiv  Detail & Related papers  (2026-04-14T14:17:38Z)
• Efficient Source-Free Time-Series Adaptation via Parameter Subspace Disentanglement [0.7558576228782637]
  We propose a framework for efficient Source-Free Domain Adaptation (SFDA)<n>Our approach introduces an improved paradigm for source-model preparation and target-side adaptation.<n>We demonstrate that our framework is compatible with various SFDA methods and achieves significant computational efficiency.
  arXiv  Detail & Related papers  (2024-10-03T02:12:03Z)
• Parameter-Efficient Fine-Tuning With Adapters [5.948206235442328]
  This research introduces a novel adaptation method utilizing the UniPELT framework as a base. Our method employs adapters, which enable efficient transfer of pretrained models to new tasks with minimal retraining of the base model parameters.
  arXiv  Detail & Related papers  (2024-05-09T01:40:38Z)
• Dynamic Tuning Towards Parameter and Inference Efficiency for ViT Adaptation [67.13876021157887]
  Dynamic Tuning (DyT) is a novel approach to improve both parameter and inference efficiency for ViT adaptation. DyT achieves superior performance compared to existing PEFT methods while evoking only 71% of their FLOPs on the VTAB-1K benchmark.
  arXiv  Detail & Related papers  (2024-03-18T14:05:52Z)
• Dynamic Adapter Meets Prompt Tuning: Parameter-Efficient Transfer Learning for Point Cloud Analysis [51.14136878142034]
  Point cloud analysis has achieved outstanding performance by transferring point cloud pre-trained models. Existing methods for model adaptation usually update all model parameters, which is inefficient as it relies on high computational costs. In this paper, we aim to study parameter-efficient transfer learning for point cloud analysis with an ideal trade-off between task performance and parameter efficiency.
  arXiv  Detail & Related papers  (2024-03-03T08:25:04Z)
• Advancing Parameter Efficiency in Fine-tuning via Representation Editing [41.81020951061438]
  We propose a novel fine-tuning approach for neural models, named Representation EDiting (RED) RED modifies the representations generated at some layers through the application of scaling and biasing operations. Remarkably, RED achieves results comparable or superior to both full parameter fine-tuning and other PEFT methods.
  arXiv  Detail & Related papers  (2024-02-23T08:21:02Z)
• Parameter-Efficient Fine-Tuning for Pre-Trained Vision Models: A Survey and Benchmark [95.0484665958967]
  Pre-trained vision models (PVMs) have demonstrated remarkable adaptability across a wide range of downstream vision tasks.<n>As these models scale to billions or even trillions of parameters, conventional full fine-tuning has become increasingly impractical due to its high computational and storage demands.<n> parameter-efficient fine-tuning (PEFT) has emerged as a promising alternative, aiming to achieve performance comparable to full fine-tuning while making minimal adjustments to the model parameters.
  arXiv  Detail & Related papers  (2024-02-03T19:12:20Z)
• When Parameter-efficient Tuning Meets General-purpose Vision-language Models [65.19127815275307]
  PETAL revolutionizes the training process by requiring only 0.5% of the total parameters, achieved through a unique mode approximation technique. Our experiments reveal that PETAL not only outperforms current state-of-the-art methods in most scenarios but also surpasses full fine-tuning models in effectiveness.
  arXiv  Detail & Related papers  (2023-12-16T17:13:08Z)
• E^2VPT: An Effective and Efficient Approach for Visual Prompt Tuning [55.50908600818483]
  Fine-tuning large-scale pretrained vision models for new tasks has become increasingly parameter-intensive. We propose an Effective and Efficient Visual Prompt Tuning (E2VPT) approach for large-scale transformer-based model adaptation. Our approach outperforms several state-of-the-art baselines on two benchmarks.
  arXiv  Detail & Related papers  (2023-07-25T19:03:21Z)
• Empirical Analysis of the Strengths and Weaknesses of PEFT Techniques for LLMs [1.867982979635437]
  We provide a benchmark of various PEFT techniques and evaluate model performance across different data scales. Contrary to popular belief, we empirically prove that PEFT techniques converge slower than full tuning in low data scenarios. We further optimize these PEFT techniques by selectively choosing which parts of the model to train, and find that these techniques can be applied with significantly fewer parameters.
  arXiv  Detail & Related papers  (2023-04-28T17:39:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.