A Semantic-based Layer Freezing Approach to Efficient Fine-Tuning of Language Models

• URL: http://arxiv.org/abs/2406.11753v1
• Date: Mon, 17 Jun 2024 17:13:08 GMT
• Title: A Semantic-based Layer Freezing Approach to Efficient Fine-Tuning of Language Models
• Authors: Jian Gu, Aldeida Aleti, Chunyang Chen, Hongyu Zhang,
• Abstract summary: Finetuning language models (LMs) is crucial for adapting the models to downstream data and tasks. Existing work, such as parameter-efficient finetuning (PEFT), often focuses on textithow to finetune but neglects the issue of textitwhere to finetune
• Score: 32.178931149612644
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Finetuning language models (LMs) is crucial for adapting the models to downstream data and tasks. However, full finetuning is usually costly. Existing work, such as parameter-efficient finetuning (PEFT), often focuses on \textit{how to finetune} but neglects the issue of \textit{where to finetune}. As a pioneering work on answering where to finetune (at the layer level), we conduct a semantic analysis of the LM inference process. We first propose a virtual transition of the latent representation and then trace its factual transition. Based on the deviation in transitions, we estimate the gain of finetuning each model layer, and further, narrow down the scope for finetuning. We perform extensive experiments across well-known LMs and datasets. The results show that our approach is effective and efficient, and outperforms the existing baselines. Our approach is orthogonal to existing efficient techniques, such as PEFT methods, offering practical values on LM finetuning.

Related papers

• PAFT: A Parallel Training Paradigm for Effective LLM Fine-Tuning [17.73193523921637]
  Large language models (LLMs) have shown remarkable abilities in diverse natural language processing (NLP) tasks. LLMs generally undergo supervised fine-tuning (SFT) followed by preference alignment to be usable in downstream applications. This paper introduces PAFT, a new PArallel training paradigm for effective LLM Fine-Tuning.
  arXiv  Detail & Related papers  (2024-06-25T20:11:37Z)
• Comparative Analysis of Different Efficient Fine Tuning Methods of Large Language Models (LLMs) in Low-Resource Setting [0.0]
  We try to push the understanding of different fine-tuning strategies for large language models (LLMs) We compare state-of-the-art methods like vanilla fine-tuning and Pattern-Based Fine-Tuning (PBFT) on pre-trained models across two datasets, COLA and MNLI. Our findings suggest that these alternative strategies can exhibit out-of-domain generalization comparable to that of vanilla FT and PBFT.
  arXiv  Detail & Related papers  (2024-05-21T20:08:52Z)
• When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method [56.571951345048355]
  Large language models (LLMs) often adopt finetuning to unlock their capabilities for downstream applications. We study whether and how different scaling factors, including LLM model size, pretraining data size, new finetuning parameter size and finetuning data size, affect the finetuning performance.
  arXiv  Detail & Related papers  (2024-02-27T04:18:49Z)
• AutoFT: Learning an Objective for Robust Fine-Tuning [60.641186718253735]
  Foundation models encode rich representations that can be adapted to downstream tasks by fine-tuning. Current approaches to robust fine-tuning use hand-crafted regularization techniques. We propose AutoFT, a data-driven approach for robust fine-tuning.
  arXiv  Detail & Related papers  (2024-01-18T18:58:49Z)
• Sparse is Enough in Fine-tuning Pre-trained Large Language Models [98.46493578509039]
  We propose a gradient-based sparse fine-tuning algorithm, named Sparse Increment Fine-Tuning (SIFT) We validate its effectiveness on a range of tasks including the GLUE Benchmark and Instruction-tuning.
  arXiv  Detail & Related papers  (2023-12-19T06:06:30Z)
• Parameter-Efficient Orthogonal Finetuning via Butterfly Factorization [102.92240148504774]
  We study a principled finetuning paradigm -- Orthogonal Finetuning (OFT) -- for downstream task adaptation. Despite demonstrating good generalizability, OFT still uses a fairly large number of trainable parameters. We apply this parameterization to OFT, creating a novel parameter-efficient finetuning method, called Orthogonal Butterfly (BOFT)
  arXiv  Detail & Related papers  (2023-11-10T18:59:54Z)
• Ahead-of-Time P-Tuning [0.2538209532048867]
  Ahead-of-Time (AoT) P-Tuning is a parameter-efficient fine-tuning method for pre-trained Language Models (LMs) We evaluate AoT P-Tuning on GLUE and SuperGLUE benchmarking datasets using RoBERTa and DeBERTa models. Our method enables multi-task inference with a single backbone LM, making it a practical solution for real-world applications.
  arXiv  Detail & Related papers  (2023-05-18T09:24:53Z)
• Trainable Projected Gradient Method for Robust Fine-tuning [36.470333094917436]
  We propose Trainable Projected Gradient Method (TPGM) to automatically learn the constraint imposed for each layer for a fine-grained fine-tuning regularization. This is motivated by formulating fine-tuning as a bi-level constrained optimization problem. We show that TPGM outperforms existing fine-tuning methods in OOD performance while matching the best in-distribution (ID) performance.
  arXiv  Detail & Related papers  (2023-03-19T17:30:44Z)
• Scaling & Shifting Your Features: A New Baseline for Efficient Model Tuning [126.84770886628833]
  Existing finetuning methods either tune all parameters of the pretrained model (full finetuning) or only tune the last linear layer (linear probing) We propose a new parameter-efficient finetuning method termed as SSF, representing that researchers only need to Scale and Shift the deep Features extracted by a pre-trained model to catch up with the performance full finetuning.
  arXiv  Detail & Related papers  (2022-10-17T08:14:49Z)
• Model-Agnostic Multitask Fine-tuning for Few-shot Vision-Language Transfer Learning [59.38343286807997]
  We propose Model-Agnostic Multitask Fine-tuning (MAMF) for vision-language models on unseen tasks. Compared with model-agnostic meta-learning (MAML), MAMF discards the bi-level optimization and uses only first-order gradients. We show that MAMF consistently outperforms the classical fine-tuning method for few-shot transfer learning on five benchmark datasets.
  arXiv  Detail & Related papers  (2022-03-09T17:26:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.