Transfer learning optimization based on evolutionary selective fine tuning

• URL: http://arxiv.org/abs/2508.15367v1
• Date: Thu, 21 Aug 2025 08:51:43 GMT
• Title: Transfer learning optimization based on evolutionary selective fine tuning
• Authors: Jacinto Colan, Ana Davila, Yasuhisa Hasegawa,
• Abstract summary: Transfer learning offers a strategy for adapting pre-trained models to new tasks.<n>Traditional fine-tuning often involves updating all model parameters.<n>BioTune selectively fine-tunes layers to enhance transfer learning efficiency.
• Score: 2.271776292902496
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep learning has shown substantial progress in image analysis. However, the computational demands of large, fully trained models remain a consideration. Transfer learning offers a strategy for adapting pre-trained models to new tasks. Traditional fine-tuning often involves updating all model parameters, which can potentially lead to overfitting and higher computational costs. This paper introduces BioTune, an evolutionary adaptive fine-tuning technique that selectively fine-tunes layers to enhance transfer learning efficiency. BioTune employs an evolutionary algorithm to identify a focused set of layers for fine-tuning, aiming to optimize model performance on a given target task. Evaluation across nine image classification datasets from various domains indicates that BioTune achieves competitive or improved accuracy and efficiency compared to existing fine-tuning methods such as AutoRGN and LoRA. By concentrating the fine-tuning process on a subset of relevant layers, BioTune reduces the number of trainable parameters, potentially leading to decreased computational cost and facilitating more efficient transfer learning across diverse data characteristics and distributions.

Related papers

• Bio-inspired fine-tuning for selective transfer learning in image classification [1.1371756033920992]
  We introduce BioTune, a novel adaptive fine-tuning technique utilizing evolutionary optimization.<n>BioTune enhances transfer learning by optimally choosing which layers to freeze and adjusting learning rates for unfrozen layers.<n>BioTune consistently achieves top performance across four different CNN architectures.
  arXiv  Detail & Related papers  (2026-01-16T12:28:49Z)
• Optimizing ML Training with Metagradient Descent [69.89631748402377]
  We introduce an algorithm for efficiently calculating metagradients -- gradients through model training -- at scale.<n>We then introduce a "smooth model training" framework that enables effective optimization using metagradients.
  arXiv  Detail & Related papers  (2025-03-17T22:18:24Z)
• Efficient Transferability Assessment for Selection of Pre-trained Detectors [63.21514888618542]
  This paper studies the efficient transferability assessment of pre-trained object detectors. We build up a detector transferability benchmark which contains a large and diverse zoo of pre-trained detectors. Experimental results demonstrate that our method outperforms other state-of-the-art approaches in assessing transferability.
  arXiv  Detail & Related papers  (2024-03-14T14:23:23Z)
• Diffusion-Based Neural Network Weights Generation [80.89706112736353]
  D2NWG is a diffusion-based neural network weights generation technique that efficiently produces high-performing weights for transfer learning. Our method extends generative hyper-representation learning to recast the latent diffusion paradigm for neural network weights generation. Our approach is scalable to large architectures such as large language models (LLMs), overcoming the limitations of current parameter generation techniques.
  arXiv  Detail & Related papers  (2024-02-28T08:34:23Z)
• Gradual Optimization Learning for Conformational Energy Minimization [69.36925478047682]
  Gradual Optimization Learning Framework (GOLF) for energy minimization with neural networks significantly reduces the required additional data. Our results demonstrate that the neural network trained with GOLF performs on par with the oracle on a benchmark of diverse drug-like molecules.
  arXiv  Detail & Related papers  (2023-11-05T11:48:08Z)
• Towards Compute-Optimal Transfer Learning [82.88829463290041]
  We argue that zero-shot structured pruning of pretrained models allows them to increase compute efficiency with minimal reduction in performance. Our results show that pruning convolutional filters of pretrained models can lead to more than 20% performance improvement in low computational regimes.
  arXiv  Detail & Related papers  (2023-04-25T21:49:09Z)
• Surgical Fine-Tuning Improves Adaptation to Distribution Shifts [114.17184775397067]
  A common approach to transfer learning under distribution shift is to fine-tune the last few layers of a pre-trained model. This paper shows that in such settings, selectively fine-tuning a subset of layers matches or outperforms commonly used fine-tuning approaches.
  arXiv  Detail & Related papers  (2022-10-20T17:59:15Z)
• Top-Tuning: a study on transfer learning for an efficient alternative to fine tuning for image classification with fast kernel methods [12.325059377851485]
  In this paper, we consider a simple transfer learning approach exploiting pre-trained convolutional features as input for a fast-to-train kernel method. We show that the top-tuning approach provides comparable accuracy with respect to fine-tuning, with a training time between one and two orders of magnitude smaller.
  arXiv  Detail & Related papers  (2022-09-16T13:46:59Z)
• EvoPruneDeepTL: An Evolutionary Pruning Model for Transfer Learning based Deep Neural Networks [15.29595828816055]
  We propose an evolutionary pruning model for Transfer Learning based Deep Neural Networks. EvoPruneDeepTL replaces the last fully-connected layers with sparse layers optimized by a genetic algorithm. Results show the contribution of EvoPruneDeepTL and feature selection to the overall computational efficiency of the network.
  arXiv  Detail & Related papers  (2022-02-08T13:07:55Z)
• Zoo-Tuning: Adaptive Transfer from a Zoo of Models [82.9120546160422]
  Zoo-Tuning learns to adaptively transfer the parameters of pretrained models to the target task. We evaluate our approach on a variety of tasks, including reinforcement learning, image classification, and facial landmark detection.
  arXiv  Detail & Related papers  (2021-06-29T14:09:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.