Histogram-based Parameter-efficient Tuning for Passive Sonar Classification

• URL: http://arxiv.org/abs/2504.15214v2
• Date: Tue, 22 Apr 2025 16:24:58 GMT
• Title: Histogram-based Parameter-efficient Tuning for Passive Sonar Classification
• Authors: Amirmohammad Mohammadi, Davelle Carreiro, Alexandra Van Dine, Joshua Peeples,
• Abstract summary: We propose a novel parameter-efficient tuning (HPT) technique that captures statistics of the target domain and modulates the embeddings.<n> Experimental results on three downstream passive sonar datasets (ShipsEar, DeepShip, VTUAD) demonstrate that HPT outperforms conventional adapters.
• Score: 42.23422932643755
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Parameter-efficient transfer learning (PETL) methods adapt large artificial neural networks to downstream tasks without fine-tuning the entire model. However, existing additive methods, such as adapters, sometimes struggle to capture distributional shifts in intermediate feature embeddings. We propose a novel histogram-based parameter-efficient tuning (HPT) technique that captures the statistics of the target domain and modulates the embeddings. Experimental results on three downstream passive sonar datasets (ShipsEar, DeepShip, VTUAD) demonstrate that HPT outperforms conventional adapters. Notably, HPT achieves 91.8% vs. 89.8% accuracy on VTUAD. Furthermore, HPT trains faster and yields feature representations closer to those of fully fine-tuned models. Overall, HPT balances parameter savings and performance, providing a distribution-aware alternative to existing adapters and shows a promising direction for scalable transfer learning in resource-constrained environments. The code is publicly available: https://github.com/Advanced-Vision-and-Learning-Lab/HLAST_DeepShip_ParameterEfficient.

Related papers

• Optimizing Specific and Shared Parameters for Efficient Parameter Tuning [46.57365875007367]
  We propose SaS, a novel PETL method that effectively mitigates distributional shifts during fine-tuning. SaS captures common statistical characteristics across layers using low-rank projections. Experiments on diverse downstream tasks, few-shot settings and domain generalization demonstrate that SaS significantly enhances performance.
  arXiv  Detail & Related papers  (2025-04-04T13:43:54Z)
• Densely Connected Parameter-Efficient Tuning for Referring Image Segmentation [30.912818564963512]
  DETRIS is a parameter-efficient tuning framework designed to enhance low-rank visual feature propagation. Our simple yet efficient approach greatly surpasses state-of-the-art methods with 0.9% to 1.8% backbone parameter updates.
  arXiv  Detail & Related papers  (2025-01-15T05:00:03Z)
• Skip Tuning: Pre-trained Vision-Language Models are Effective and Efficient Adapters Themselves [123.07450481623124]
  We propose Skip Tuning as a novel paradigm for adapting vision-language models to downstream tasks. Unlike existing PT or adapter-based methods, Skip Tuning applies Layer-wise Skipping (LSkip) and Class-wise Skipping (CSkip) upon the FT baseline without introducing extra context vectors or adapter modules.
  arXiv  Detail & Related papers  (2024-12-16T07:33:23Z)
• ALoRE: Efficient Visual Adaptation via Aggregating Low Rank Experts [71.91042186338163]
  ALoRE is a novel PETL method that reuses the hypercomplex parameterized space constructed by Kronecker product to Aggregate Low Rank Experts.<n>Thanks to the artful design, ALoRE maintains negligible extra parameters and can be effortlessly merged into the frozen backbone.
  arXiv  Detail & Related papers  (2024-12-11T12:31:30Z)
• Test-time adaptation for geospatial point cloud semantic segmentation with distinct domain shifts [6.80671668491958]
  Test-time adaptation (TTA) allows direct adaptation of a pre-trained model to unlabeled data during inference stage without access to source data or additional training. We propose three domain shift paradigms: photogrammetric to airborne LiDAR, airborne to mobile LiDAR, and synthetic to mobile laser scanning. Experimental results show our method improves classification accuracy by up to 20% mIoU, outperforming other methods.
  arXiv  Detail & Related papers  (2024-07-08T15:40:28Z)
• REP: Resource-Efficient Prompting for Rehearsal-Free Continual Learning [23.92661395403251]
  Recent rehearsal-free methods, guided by prompts, excel in vision-related continual learning (CL) with drifting data but lack resource efficiency.<n>We introduce Resource-Efficient Prompting (REP), which improves the computational and memory efficiency of prompt-based rehearsal-free methods.<n>Our approach employs swift prompt selection to refine input data using a carefully provisioned model.
  arXiv  Detail & Related papers  (2024-06-07T09:17:33Z)
• Parameter-Efficient Fine-Tuning With Adapters [5.948206235442328]
  This research introduces a novel adaptation method utilizing the UniPELT framework as a base. Our method employs adapters, which enable efficient transfer of pretrained models to new tasks with minimal retraining of the base model parameters.
  arXiv  Detail & Related papers  (2024-05-09T01:40:38Z)
• Dynamic Tuning Towards Parameter and Inference Efficiency for ViT Adaptation [67.13876021157887]
  Dynamic Tuning (DyT) is a novel approach to improve both parameter and inference efficiency for ViT adaptation. DyT achieves superior performance compared to existing PEFT methods while evoking only 71% of their FLOPs on the VTAB-1K benchmark.
  arXiv  Detail & Related papers  (2024-03-18T14:05:52Z)
• Dynamic Adapter Meets Prompt Tuning: Parameter-Efficient Transfer Learning for Point Cloud Analysis [51.14136878142034]
  Point cloud analysis has achieved outstanding performance by transferring point cloud pre-trained models. Existing methods for model adaptation usually update all model parameters, which is inefficient as it relies on high computational costs. In this paper, we aim to study parameter-efficient transfer learning for point cloud analysis with an ideal trade-off between task performance and parameter efficiency.
  arXiv  Detail & Related papers  (2024-03-03T08:25:04Z)
• Adaptive Sparsity Level during Training for Efficient Time Series Forecasting with Transformers [20.23085795744602]
  We propose textbfAdaptive textbfSparsity textbfLevel (textbfPALS) to automatically seek a decent balance between loss and sparsity. PALS draws inspiration from sparse training and during-training methods. It introduces the novel "expand" mechanism in training sparse neural networks, allowing the model to dynamically shrink, expand, or remain stable to find a proper sparsity level.
  arXiv  Detail & Related papers  (2023-05-28T06:57:27Z)
• Med-Tuning: A New Parameter-Efficient Tuning Framework for Medical Volumetric Segmentation [37.42382366505377]
  We introduce a new framework named Med-Tuning to realize parameter-efficient tuning (PET) for medical volumetric segmentation task. Our framework enhances the 2D baselines's precision on segmentation tasks, which are pre-trained on natural images. Compared to full FT, Med-Tuning reduces the fine-tuned model parameters by up to 4x, with even better segmentation performance.
  arXiv  Detail & Related papers  (2023-04-21T10:47:13Z)
• Sensitivity-Aware Visual Parameter-Efficient Fine-Tuning [91.5113227694443]
  We propose a novel visual. sensuous-aware fine-Tuning (SPT) scheme. SPT allocates trainable parameters to task-specific important positions. Experiments on a wide range of downstream recognition tasks show that our SPT is complementary to the existing PEFT methods.
  arXiv  Detail & Related papers  (2023-03-15T12:34:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.