Self-Auditing Parameter-Efficient Fine-Tuning for Few-Shot 3D Medical Image Segmentation

• URL: http://arxiv.org/abs/2603.05822v1
• Date: Fri, 06 Mar 2026 02:17:12 GMT
• Title: Self-Auditing Parameter-Efficient Fine-Tuning for Few-Shot 3D Medical Image Segmentation
• Abstract summary: SEA-PEFT treats adapter configuration as an online allocation problem solved during fine-tuning.<n>SEA-PEFT uses a search-audit-al loop that trains active adapters, estimates each adapter's Dice utility by momentarily toggling it off, and then reselects the active set under a parameter budget.<n>On TotalSegmentator and FLARE'22, SEA-PEFT improves mean Dice by 2.4--2.8 points over the strongest fixed-topology PEFT smoothing.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Adapting foundation models to new clinical sites remains challenging in practice. Domain shift and scarce annotations must be handled by experts, yet many clinical groups do not have ready access to skilled AI engineers to tune adapter designs and training recipes. As a result, adaptation cycles can stretch from weeks to months, particularly in few-shot settings. Existing PEFT methods either require manual adapter configuration or automated searches that are computationally infeasible in few-shot 3D settings. We propose SEA-PEFT (SElf-Auditing Parameter-Efficient Fine-Tuning) to automate this process. SEA-PEFT treats adapter configuration as an online allocation problem solved during fine-tuning rather than through manual, fixed-topology choices. SEA-PEFT uses a search-audit-allocate loop that trains active adapters, estimates each adapter's Dice utility by momentarily toggling it off, and then reselects the active set under a parameter budget using a greedy knapsack allocator. Exponential Moving Average and Interquartile Range smoothing, together with a Finite-State Ranking controller, stabilize the loop and improve reliability in high-noise few-shot regimes. On TotalSegmentator and FLARE'22, SEA-PEFT improves mean Dice by 2.4--2.8 points over the strongest fixed-topology PEFT baselines across 1/5/10-shot settings while training <1% of parameters. For reproducibility purposes, we made our code publicly available at https://github.com/tsly123/SEA_PEFT

Related papers

• Circuit Fine-Tuning for Compute-Efficient Transformer Adaptation [3.52359746858894]
  Circuit Fine-Tuning (CFT) is a compute-efficient framework that uses circuit discovery to select modules for fine-tuning before training.<n>CFT needs no learning-rate warmup and reaches peak accuracy in $sim20$ epochs on average.
  arXiv  Detail & Related papers  (2026-08-08T21:04:19Z)
• Fisher-Guided Progressive Parameter Selection for Adaptive Fine-Tuning [5.8006596438443125]
  We introduce textbfFisherAdapTune, a Fisher-guided Adaptive Fine-Tuning framework that selects parameter groups by tracking the temporal drift of their Fisher geometry.<n>We evaluate our approach on a downstream segmentation task, and results show FisherAdapTune improves the in-distribution performance and zero-shot transfer in multiple settings.
  arXiv  Detail & Related papers  (2026-06-08T21:35:11Z)
• PreFT: Prefill-only finetuning for efficient inference [54.58291801311547]
  We develop and release an efficient implementation of two prefill-only PEFTs, LoRA and ReFT, on the vLLM inference engine.<n>We show that serving multi-user PreFTs is more efficient than traditional PEFTs ($1.9times$ the throughput when serving $512$ adapters on Llama 3.1 70B)
  arXiv  Detail & Related papers  (2026-05-14T00:19:41Z)
• PointTPA: Dynamic Network Parameter Adaptation for 3D Scene Understanding [51.98954431091292]
  We propose a Test-time Adaptation framework that generates input-aware network parameters for scene-level point clouds.<n>PointTPA achieves 78.4% mIoU on ScanNet validation, surpassing existing parameter-efficient fine-tuning methods.
  arXiv  Detail & Related papers  (2026-04-06T17:59:58Z)
• GateRA: Token-Aware Modulation for Parameter-Efficient Fine-Tuning [51.79350934271497]
  GateRA is a unified framework that introduces token-aware modulation to dynamically adjust the strength of PEFT updates.<n>By incorporating adaptive gating into standard PEFT branches, GateRA enables selective, token-level adaptation.<n> Experiments on multiple commonsense reasoning benchmarks demonstrate that GateRA consistently outperforms or matches prior PEFT methods.
  arXiv  Detail & Related papers  (2025-11-15T17:55:47Z)
• Task-Adaptive Parameter-Efficient Fine-Tuning for Weather Foundation Models [28.79157031663951]
  We introduce WeatherPEFT, a novel PEFT framework for Weather Foundation Models (WFMs)<n>Task-Tu Dynamic Prompting (TTu) injects the embedding weights within the encoder to the input tokens of the backbone of the pre-trained via internal and external pattern extraction.<n>Fisher-Guided Adaptive Selection (SFAS) identifies the most task-critical randomness, preserving in pre-trained knowledge, but also stabilizing the selection.<n>We demonstrate the effectiveness and efficiency of WeatherPEFT on three downstream tasks, where existing PEFT methods show significant gaps versus Full-Tuning, and WeatherPEFT performance parity
  arXiv  Detail & Related papers  (2025-09-26T07:54:05Z)
• Don't be lazy: CompleteP enables compute-efficient deep transformers [50.85418589942566]
  Some parameterizations fail to transfer optimal base HPs across changes in model depth.<n>We develop theory to show parameterizations may still exist in the lazy learning regime.<n>We identify and adopt the parameterization we call CompleteP that achieves both depth-wise HP transfer and non-lazy learning in all layers.
  arXiv  Detail & Related papers  (2025-05-02T22:45:14Z)
• TT-LoRA MoE: Unifying Parameter-Efficient Fine-Tuning and Sparse Mixture-of-Experts [4.5558042369389105]
  TT-LoRA MoE decomposes training into two distinct optimized stages.<n>First, we independently train lightweight, tensorized low-rank adapters (TT-LoRA experts)<n>Subsequently, these expert adapters remain frozen, eliminating inter-task interference and forgetting in multi-task setting.<n>A sparse MoE router, trained separately, dynamically leverages base model representations to select exactly one specialized adapter per input at inference time.<n> Comprehensive experiments confirm our architecture retains the memory efficiency of low-rank adapters, seamlessly scales to large expert pools, and achieves robust task-level optimization.
  arXiv  Detail & Related papers  (2025-04-29T21:46:43Z)
• FISH-Tuning: Enhancing PEFT Methods with Fisher Information [3.9274736061387854]
  FISH Mask is a selection-based PEFT technique that identifies a critical subset of pre-trained parameters using approximate Fisher information.<n>We propose textbfFISH-Tuning, a novel approach that incorporates FISH Mask into such PEFT methods, including LoRA, Adapter, and their variants.
  arXiv  Detail & Related papers  (2025-04-05T04:05:55Z)
• Strong Baselines for Parameter Efficient Few-Shot Fine-tuning [50.83426196335385]
  Few-shot classification (FSC) entails learning novel classes given only a few examples per class after a pre-training (or meta-training) phase. Recent works have shown that simply fine-tuning a pre-trained Vision Transformer (ViT) on new test classes is a strong approach for FSC. Fine-tuning ViTs, however, is expensive in time, compute and storage. This has motivated the design of parameter efficient fine-tuning (PEFT) methods which fine-tune only a fraction of the Transformer's parameters.
  arXiv  Detail & Related papers  (2023-04-04T16:14:39Z)
• Sensitivity-Aware Visual Parameter-Efficient Fine-Tuning [91.5113227694443]
  We propose a novel visual. sensuous-aware fine-Tuning (SPT) scheme. SPT allocates trainable parameters to task-specific important positions. Experiments on a wide range of downstream recognition tasks show that our SPT is complementary to the existing PEFT methods.
  arXiv  Detail & Related papers  (2023-03-15T12:34:24Z)
• Online Hyperparameter Optimization for Class-Incremental Learning [99.70569355681174]
  Class-incremental learning (CIL) aims to train a classification model while the number of classes increases phase-by-phase. An inherent challenge of CIL is the stability-plasticity tradeoff, i.e., CIL models should keep stable to retain old knowledge and keep plastic to absorb new knowledge. We propose an online learning method that can adaptively optimize the tradeoff without knowing the setting as a priori.
  arXiv  Detail & Related papers  (2023-01-11T17:58:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.