Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey

• URL: http://arxiv.org/abs/2403.14608v7
• Date: Mon, 16 Sep 2024 02:54:50 GMT
• Title: Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey
• Authors: Zeyu Han, Chao Gao, Jinyang Liu, Jeff Zhang, Sai Qian Zhang,
• Abstract summary: Efficient Fine-Tuning (PEFT) provides a practical solution by efficiently adjusting the large models over the various downstream tasks. PEFT refers to the process of adjusting the parameters of a pre-trained large model to adapt it to a specific task or domain. We present comprehensive studies of various PEFT algorithms, examining their performance and computational overhead.
• Score: 18.00772798876708
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large models represent a groundbreaking advancement in multiple application fields, enabling remarkable achievements across various tasks. However, their unprecedented scale comes with significant computational costs. These models, often consisting of billions of parameters, require vast amounts of computational resources for execution. Especially, the expansive scale and computational demands pose considerable challenges when customizing them for particular downstream tasks, particularly over the hardware platforms constrained by computational capabilities. Parameter Efficient Fine-Tuning (PEFT) provides a practical solution by efficiently adjusting the large models over the various downstream tasks. In particular, PEFT refers to the process of adjusting the parameters of a pre-trained large model to adapt it to a specific task or domain while minimizing the number of additional parameters introduced or computational resources required. This approach is particularly important when dealing with large-scale language models with high parameter counts, as fine-tuning these models from scratch can be computationally expensive and resource-intensive, posing considerable challenges in the supporting system platform design. In this survey, we present comprehensive studies of various PEFT algorithms, examining their performance and computational overhead. Moreover, we provide an overview of applications developed using different PEFT algorithms and discuss common techniques employed to mitigate computation costs for PEFT. In addition to providing an extensive survey from an algorithmic standpoint, we also examine various real-world system designs to investigate the implementation costs associated with different PEFT approaches. This survey serves as a valuable resource for researchers aiming to understand both the PEFT algorithm and its system implementation, offering detailed ......

Related papers

• Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologies [17.904502959675337]
  Large models require substantial computational resources and GPU memory to operate. Efficient Fine-Tuning (PEFT) offers a practical solution by efficiently adjusting the parameters of large pre-trained models to suit various downstream tasks. This review mainly introduces the preliminary knowledge of PEFT, the core ideas and principles of various PEFT algorithms, the applications of PEFT, and potential future research directions.
  arXiv  Detail & Related papers  (2024-10-24T13:58:59Z)
• Enhancing Multi-Step Reasoning Abilities of Language Models through Direct Q-Function Optimization [50.485788083202124]
  Reinforcement Learning (RL) plays a crucial role in aligning large language models with human preferences and improving their ability to perform complex tasks. We introduce Direct Q-function Optimization (DQO), which formulates the response generation process as a Markov Decision Process (MDP) and utilizes the soft actor-critic (SAC) framework to optimize a Q-function directly parameterized by the language model. Experimental results on two math problem-solving datasets, GSM8K and MATH, demonstrate that DQO outperforms previous methods, establishing it as a promising offline reinforcement learning approach for aligning language models.
  arXiv  Detail & Related papers  (2024-10-11T23:29:20Z)
• Unleashing the Power of Task-Specific Directions in Parameter Efficient Fine-tuning [65.31677646659895]
  This paper focuses on the concept of task-specific directions (TSDs)-critical for transitioning large models from pretrained states to task-specific enhancements in PEFT. We introduce a novel approach, LoRA-Dash, which aims to maximize the impact of TSDs during the fine-tuning process, thereby enhancing model performance on targeted tasks.
  arXiv  Detail & Related papers  (2024-09-02T08:10:51Z)
• Parameter Efficient Fine Tuning: A Comprehensive Analysis Across Applications [0.7421845364041001]
  The rise of deep learning has marked significant progress in fields such as computer vision, natural language processing, and medical imaging. Traditional fine-tuning methods, involving adjustments to all parameters, face challenges due to high computational and memory demands. This review examines Efficient Fine-Tuning (PEFT) techniques, which selectively update parameters to balance computational efficiency with performance.
  arXiv  Detail & Related papers  (2024-04-21T02:26:15Z)
• Efficiency at Scale: Investigating the Performance of Diminutive Language Models in Clinical Tasks [2.834743715323873]
  We present an investigation into the suitability of different PEFT methods to clinical decision-making tasks. Our analysis shows that the performance of most PEFT approaches varies significantly from one task to another. The effectiveness of PEFT methods in the clinical domain is evident, particularly for specialised models which can operate on low-cost, in-house computing infrastructure.
  arXiv  Detail & Related papers  (2024-02-16T11:30:11Z)
• Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessment [12.674032145667763]
  We present a comprehensive and systematic review of Efficient Fine-Tuning (PEFT) methods for pretrained language models (PLMs) PEFT offers an effective solution by reducing the number of fine-tuning parameters and memory usage while achieving comparable performance to full fine-tuning. We conduct experiments using several representative PEFT methods to better understand their effectiveness in parameter efficiency and memory efficiency.
  arXiv  Detail & Related papers  (2023-12-19T13:31:24Z)
• The Efficiency Spectrum of Large Language Models: An Algorithmic Survey [54.19942426544731]
  The rapid growth of Large Language Models (LLMs) has been a driving force in transforming various domains. This paper examines the multi-faceted dimensions of efficiency essential for the end-to-end algorithmic development of LLMs.
  arXiv  Detail & Related papers  (2023-12-01T16:00:25Z)
• Design Automation for Fast, Lightweight, and Effective Deep Learning Models: A Survey [53.258091735278875]
  This survey covers studies of design automation techniques for deep learning models targeting edge computing. It offers an overview and comparison of key metrics that are used commonly to quantify the proficiency of models in terms of effectiveness, lightness, and computational costs. The survey proceeds to cover three categories of the state-of-the-art of deep model design automation techniques.
  arXiv  Detail & Related papers  (2022-08-22T12:12:43Z)
• How Much More Data Do I Need? Estimating Requirements for Downstream Tasks [99.44608160188905]
  Given a small training data set and a learning algorithm, how much more data is necessary to reach a target validation or test performance? Overestimating or underestimating data requirements incurs substantial costs that could be avoided with an adequate budget. Using our guidelines, practitioners can accurately estimate data requirements of machine learning systems to gain savings in both development time and data acquisition costs.
  arXiv  Detail & Related papers  (2022-07-04T21:16:05Z)
• Deep Multi-Fidelity Active Learning of High-dimensional Outputs [17.370056935194786]
  We develop a deep neural network-based multi-fidelity model for learning with high-dimensional outputs. We then propose a mutual information-based acquisition function that extends the predictive entropy principle. We show the advantage of our method in several applications of computational physics and engineering design.
  arXiv  Detail & Related papers  (2020-12-02T00:02:31Z)
• A Survey on Large-scale Machine Learning [67.6997613600942]
  Machine learning can provide deep insights into data, allowing machines to make high-quality predictions. Most sophisticated machine learning approaches suffer from huge time costs when operating on large-scale data. Large-scale Machine Learning aims to learn patterns from big data with comparable performance efficiently.
  arXiv  Detail & Related papers  (2020-08-10T06:07:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.