Flexora: Flexible Low Rank Adaptation for Large Language Models

• URL: http://arxiv.org/abs/2408.10774v3
• Date: Tue, 18 Feb 2025 13:53:51 GMT
• Title: Flexora: Flexible Low Rank Adaptation for Large Language Models
• Authors: Chenxing Wei, Yao Shu, Ying Tiffany He, Fei Richard Yu,
• Abstract summary: Large Language Models (LLMs) are driving advancements in artificial intelligence by increasing the scale of model parameters. Their performance in specific downstream tasks is usually hindered by their knowledge boundaries on these tasks. We propose the flexible low rank adaptation (Flexora) method to automatically and flexibly select the most important layers.
• Score: 12.696136981847438
• License:
• Abstract: Large Language Models (LLMs) are driving advancements in artificial intelligence by increasing the scale of model parameters, which has significantly enhanced generalization ability and unlocked new capabilities in practice. However, their performance in specific downstream tasks is usually hindered by their knowledge boundaries on these tasks. Thus, fine-tuning techniques, especially the widely used Low-Rank Adaptation (LoRA) method, have been introduced to expand the boundaries on these tasks, whereas LoRA would underperform on certain tasks owing to its potential overfitting on these tasks. To overcome this overfitting and improve the performance of LoRA, we propose the flexible low rank adaptation (Flexora) method to automatically and flexibly select the most important layers needing to be fine-tuned to achieve the best performance on different downstream tasks. Specifically, Flexora firstly frames this layer selection problem as a well-defined hyperparameter optimization (HPO) problem, then addresses it using the unrolled differentiation (UD) method, and finally selects the most useful layers based on the optimized hyperparameters. Our extensive experiments on many pretrained models and natural language tasks show that Flexora is able to consistently improve over the existing baselines, indicating the effectiveness of our Flexora in practice. We additionally provide insightful theoretical results and many ablation studies to deliver a comprehensive understanding of our Flexora.

Related papers

• ASLoRA: Adaptive Sharing Low-Rank Adaptation Across Layers [37.77593687901923]
  ASLoRA is a cross-layer parameter-sharing strategy combining global sharing with partial adaptive sharing. We conduct experiments on various NLP tasks, showing that ASLoRA outperforms LoRA while using less than 25% of the parameters.
  arXiv  Detail & Related papers  (2024-12-13T13:32:13Z)
• Learn from Downstream and Be Yourself in Multimodal Large Language Model Fine-Tuning [104.27224674122313]
  Fine-tuning MLLM has become a common practice to improve performance on specific downstream tasks. To balance the trade-off between generalization and specialization, we propose measuring the parameter importance for both pre-trained and fine-tuning distributions.
  arXiv  Detail & Related papers  (2024-11-17T01:16:37Z)
• Less is More: Extreme Gradient Boost Rank-1 Adaption for Efficient Finetuning of LLMs [75.11449420928139]
  Fine-tuning Large Language Models (LLMs) has become a crucial technique for adapting pre-trained models to downstream tasks. Low-Rank Adaptation (LoRA) has emerged as a promising solution, but there exists a gap between the practical performance of low-rank adaptations and its theoretical optimum. We propose eXtreme Gradient Boosting LoRA, a novel framework that bridges this gap by leveraging the power of ensemble learning.
  arXiv  Detail & Related papers  (2024-10-25T17:07:13Z)
• Layer-wise Importance Matters: Less Memory for Better Performance in Parameter-efficient Fine-tuning of Large Language Models [19.163639128631534]
  Importance-aware Sparse Tuning (IST) is a plug-and-play technique compatible with various PEFT methods that operate on a per-layer basis. IST dynamically updates selected layers in PEFT modules, leading to reduced memory demands.
  arXiv  Detail & Related papers  (2024-10-15T16:53:26Z)
• Lighter And Better: Towards Flexible Context Adaptation For Retrieval Augmented Generation [32.26885597587913]
  We introduce a novel approach called FlexRAG (Flexible Context Adaptation for RAG) In this approach, the retrieved contexts are compressed into compact embeddings before being encoded by the Large Language Models (LLMs) A key feature of FlexRAG is its flexibility, which enables effective support for diverse compression ratios and selective preservation of important contexts.
  arXiv  Detail & Related papers  (2024-09-24T03:25:36Z)
• Expressive and Generalizable Low-rank Adaptation for Large Models via Slow Cascaded Learning [55.5715496559514]
  LoRA Slow Cascade Learning (LoRASC) is an innovative technique designed to enhance LoRA's expressiveness and generalization capabilities. Our approach augments expressiveness through a cascaded learning strategy that enables a mixture-of-low-rank adaptation, thereby increasing the model's ability to capture complex patterns.
  arXiv  Detail & Related papers  (2024-07-01T17:28:59Z)
• One Token Can Help! Learning Scalable and Pluggable Virtual Tokens for Retrieval-Augmented Large Language Models [67.49462724595445]
  Retrieval-augmented generation (RAG) is a promising way to improve large language models (LLMs) We propose a novel method that involves learning scalable and pluggable virtual tokens for RAG.
  arXiv  Detail & Related papers  (2024-05-30T03:44:54Z)
• Entropy-Regularized Token-Level Policy Optimization for Language Agent Reinforcement [67.1393112206885]
  Large Language Models (LLMs) have shown promise as intelligent agents in interactive decision-making tasks. We introduce Entropy-Regularized Token-level Policy Optimization (ETPO), an entropy-augmented RL method tailored for optimizing LLMs at the token level. We assess the effectiveness of ETPO within a simulated environment that models data science code generation as a series of multi-step interactive tasks.
  arXiv  Detail & Related papers  (2024-02-09T07:45:26Z)
• PRILoRA: Pruned and Rank-Increasing Low-Rank Adaptation [65.268245109828]
  We introduce PRILoRA, which linearly allocates a different rank for each layer, in an increasing manner, and performs pruning throughout the training process. We validate the effectiveness of PRILoRA through extensive experiments on eight GLUE benchmarks, setting a new state of the art.
  arXiv  Detail & Related papers  (2024-01-20T20:25:17Z)
• One-for-All: Generalized LoRA for Parameter-Efficient Fine-tuning [34.109808214968176]
  Generalized LoRA (GLoRA) is an advanced approach for universal parameter-efficient fine-tuning tasks. It employs a generalized prompt module to optimize pre-trained model weights and adjust intermediate activations. GLoRA exhibits strong transfer learning, few-shot learning and domain generalization abilities.
  arXiv  Detail & Related papers  (2023-06-13T17:59:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.