From LLMs to Edge: Parameter-Efficient Fine-Tuning on Edge Devices

• URL: http://arxiv.org/abs/2507.23536v1
• Date: Thu, 31 Jul 2025 13:23:21 GMT
• Title: From LLMs to Edge: Parameter-Efficient Fine-Tuning on Edge Devices
• Authors: Georg Slamanig, Francesco Corti, Olga Saukh,
• Abstract summary: This paper benchmarks and analyzes popular PEFT methods on convolutional architectures typically deployed in resource-constrained edge environments.<n>We find that the evaluated PEFT methods are only half as memory-efficient when applied to depthwise-separable convolution architectures.
• Score: 3.4233698915405544
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Parameter-efficient fine-tuning (PEFT) methods reduce the computational costs of updating deep learning models by minimizing the number of additional parameters used to adapt a model to a down- stream task. While extensively researched in large language models (LLMs), their application to smaller models used on edge devices, such as convolutional neural networks, remains underexplored. This paper benchmarks and analyzes popular PEFT methods on convolutional architectures typically deployed in resource-constrained edge environments. We evaluate LoRA, DoRA, and GaLore for updating standard and depthwise convolutional architectures to handle distribution shifts and accommodate unseen classes. We utilize recently proposed PyTorch profilers to compare the updated model performance and computational costs of these PEFT methods with traditional fine-tuning approaches. With resource efficiency in mind, we investigate their update behavior across different rank dimensions. We find that the evaluated PEFT methods are only half as memory-efficient when applied to depthwise-separable convolution architectures, compared to their efficiency with LLMs. Conversely, when targeting convolu- tional architectures optimized for edge deployment, adapter-based PEFT methods can reduce floating point operations (FLOPs) during model updates by up to 95%. These insights offer valuable guidance for selecting PEFT methods based on hardware constraints, performance requirements, and application needs. Our code is online.

Related papers

• PrunePEFT: Iterative Hybrid Pruning for Parameter-Efficient Fine-tuning of LLMs [8.52711842775914]
  Efficient Fine-Tuning (PEFT) methods have emerged as effective and promising approaches for fine-tuning pre-trained language models.<n>In this paper, we propose a novel approach, PrunePEFT, which formulates the PEFT strategy search as a pruning problem.
  arXiv  Detail & Related papers  (2025-06-09T09:32:58Z)
• A Survey on Parameter-Efficient Fine-Tuning for Foundation Models in Federated Learning [5.280048850098648]
  Foundation models have revolutionized artificial intelligence by providing robust, versatile architectures pre-trained on large-scale datasets.<n>Adapting these massive models to specific downstream tasks requires fine-tuning, which can be prohibitively expensive in computational resources.<n>This survey provides a comprehensive review of the integration of PEFT techniques within federated learning environments.
  arXiv  Detail & Related papers  (2025-04-29T18:18:39Z)
• PLM: Efficient Peripheral Language Models Hardware-Co-Designed for Ubiquitous Computing [48.30406812516552]
  We introduce the PLM, a Peripheral Language Model, developed through a co-design process that jointly optimize model architecture and edge system constraints.<n>PLM employs a Multi-head Latent Attention mechanism and employs the squared ReLU activation function to encourage sparsity, thereby reducing peak memory footprint.<n> evaluation results demonstrate that PLM outperforms existing small language models trained on publicly available data.
  arXiv  Detail & Related papers  (2025-03-15T15:11:17Z)
• LESA: Learnable LLM Layer Scaling-Up [57.0510934286449]
  Training Large Language Models (LLMs) from scratch requires immense computational resources, making it prohibitively expensive.<n>Model scaling-up offers a promising solution by leveraging the parameters of smaller models to create larger ones.<n>We propose textbfLESA, a novel learnable method for depth scaling-up.
  arXiv  Detail & Related papers  (2025-02-19T14:58:48Z)
• Parameter-Efficient Fine-Tuning of Large Language Models for Unit Test Generation: An Empirical Study [3.5189934649278922]
  Large language models (LLMs) like GitHub Copilot struggle with real-world tasks without fine-tuning. This paper investigates full fine-tuning and various PEFT methods, including LoRA, (IA)3, and prompt tuning. Our findings show that PEFT methods can deliver performance comparable to full fine-tuning for unit test generation.
  arXiv  Detail & Related papers  (2024-11-04T09:03:18Z)
• LoRTA: Low Rank Tensor Adaptation of Large Language Models [70.32218116940393]
  Low Rank Adaptation (LoRA) is a popular Efficient Fine Tuning (PEFT) method.<n>We propose a higher-order Candecomp/Parafac (CP) decomposition, enabling a more compact and flexible representation.<n>Our method can achieve a reduction in the number of parameters while maintaining comparable performance.
  arXiv  Detail & Related papers  (2024-10-05T06:59:50Z)
• Reference Trustable Decoding: A Training-Free Augmentation Paradigm for Large Language Models [79.41139393080736]
  Large language models (LLMs) have rapidly advanced and demonstrated impressive capabilities. In-Context Learning (ICL) and. Efficient Fine-Tuning (PEFT) are currently two mainstream methods for augmenting. LLMs to downstream tasks. We propose Reference Trustable Decoding (RTD), a paradigm that allows models to quickly adapt to new tasks without fine-tuning.
  arXiv  Detail & Related papers  (2024-09-30T10:48:20Z)
• Search for Efficient Large Language Models [52.98684997131108]
  Large Language Models (LLMs) have long held sway in the realms of artificial intelligence research. Weight pruning, quantization, and distillation have been embraced to compress LLMs, targeting memory reduction and inference acceleration. Most model compression techniques concentrate on weight optimization, overlooking the exploration of optimal architectures.
  arXiv  Detail & Related papers  (2024-09-25T21:32:12Z)
• Empirical Analysis of the Strengths and Weaknesses of PEFT Techniques for LLMs [1.867982979635437]
  We provide a benchmark of various PEFT techniques and evaluate model performance across different data scales. Contrary to popular belief, we empirically prove that PEFT techniques converge slower than full tuning in low data scenarios. We further optimize these PEFT techniques by selectively choosing which parts of the model to train, and find that these techniques can be applied with significantly fewer parameters.
  arXiv  Detail & Related papers  (2023-04-28T17:39:49Z)
• Energy-efficient Task Adaptation for NLP Edge Inference Leveraging Heterogeneous Memory Architectures [68.91874045918112]
  adapter-ALBERT is an efficient model optimization for maximal data reuse across different tasks. We demonstrate the advantage of mapping the model to a heterogeneous on-chip memory architecture by performing simulations on a validated NLP edge accelerator.
  arXiv  Detail & Related papers  (2023-03-25T14:40:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.