LoRA Soups: Merging LoRAs for Practical Skill Composition Tasks

• URL: http://arxiv.org/abs/2410.13025v2
• Date: Mon, 02 Dec 2024 06:40:50 GMT
• Title: LoRA Soups: Merging LoRAs for Practical Skill Composition Tasks
• Authors: Akshara Prabhakar, Yuanzhi Li, Karthik Narasimhan, Sham Kakade, Eran Malach, Samy Jelassi,
• Abstract summary: Low-Rank Adaptation (LoRA) is a technique for parameter-efficient fine-tuning of Large Language Models (LLMs) We study how different LoRA modules can be merged to achieve skill composition.
• Score: 73.09643674975591
• License:
• Abstract: Low-Rank Adaptation (LoRA) is a popular technique for parameter-efficient fine-tuning of Large Language Models (LLMs). We study how different LoRA modules can be merged to achieve skill composition -- testing the performance of the merged model on a target task that involves combining multiple skills, each skill coming from a single LoRA. This setup is favorable when it is difficult to obtain training data for the target task and when it can be decomposed into multiple skills. First, we identify practically occurring use-cases that can be studied under the realm of skill composition, e.g. solving hard math-word problems with code, creating a bot to answer questions on proprietary manuals or about domain-specialized corpora. Our main contribution is to show that concatenation of LoRAs (CAT), which optimally weights LoRAs that were individually trained on different skills, outperforms existing model- and data- merging techniques; for instance on math-word problems, CAT beats these methods by an average of 43% and 12% respectively. Thus, this paper advocates model merging as an efficient way to solve compositional tasks and underscores CAT as a simple, compute-friendly and effective procedure. To our knowledge, this is the first work demonstrating the superiority of model merging over data mixing for binary skill composition tasks. Code and data are available at https://github.com/aksh555/LoRA-Soups

Related papers

• How Much Knowledge Can You Pack into a LoRA Adapter without Harming LLM? [55.33467849079774]
  Low-rank adaptation (LoRA) is a popular and efficient training technique for updating or domain-specific adaptation of Large Language Models. We investigate how new facts can be incorporated into the LLM using LoRA without compromising the previously learned knowledge.
  arXiv  Detail & Related papers  (2025-02-20T12:31:03Z)
• In-Context Meta LoRA Generation [61.690065588534296]
  Low-rank Adaptation (LoRA) has demonstrated remarkable capabilities for task specific fine-tuning. We propose In-Context Meta LoRA (ICM-LoRA), a novel approach that efficiently achieves task-specific customization of large language models. ICM-LoRA enables more accurate LoRA parameter reconstruction than current parameter reconstruction methods.
  arXiv  Detail & Related papers  (2025-01-29T13:12:01Z)
• CopRA: A Progressive LoRA Training Strategy [9.847045610578073]
  Low-Rank Adaptation (LoRA) is a parameter-efficient technique for fine-tuning foundation models. In this work, we propose a novel progressive training strategy for LoRA with random layer dropping. We refer to this method as Cooperative LoRA (CopRA)
  arXiv  Detail & Related papers  (2024-10-30T11:07:09Z)
• FedEx-LoRA: Exact Aggregation for Federated and Efficient Fine-Tuning of Foundation Models [5.1613368481802455]
  Low-Rank Adaptation (LoRA) is a popular technique for efficient fine-tuning of foundation models. We propose Federated Exact LoRA, or FedEx-LoRA, which adds a residual error term to the pretrained frozen weight matrix. Our approach achieves exact updates with minimal computational and communication overhead, preserving LoRA's efficiency.
  arXiv  Detail & Related papers  (2024-10-12T08:22:44Z)
• BoRA: Bayesian Hierarchical Low-Rank Adaption for Multi-Task Large Language Models [0.0]
  This paper introduces Bayesian Hierarchical Low-Rank Adaption (BoRA), a novel method for finetuning multi-task Large Language Models (LLMs) BoRA addresses trade-offs by leveraging a Bayesian hierarchical model that allows tasks to share information through global hierarchical priors. Our experimental results show that BoRA outperforms both individual and unified model approaches, achieving lower perplexity and better generalization across tasks.
  arXiv  Detail & Related papers  (2024-07-08T06:38:50Z)
• ResLoRA: Identity Residual Mapping in Low-Rank Adaption [96.59370314485074]
  We propose ResLoRA, an improved framework of low-rank adaptation (LoRA) Our method can achieve better results in fewer training steps without any extra trainable parameters or inference cost compared to LoRA. The experiments on NLG, NLU, and text-to-image tasks demonstrate the effectiveness of our method.
  arXiv  Detail & Related papers  (2024-02-28T04:33:20Z)
• LoRA-Flow: Dynamic LoRA Fusion for Large Language Models in Generative Tasks [72.88244322513039]
  LoRA employs lightweight modules to customize large language models (LLMs) for each downstream task or domain. We propose LoRA-Flow, which utilizes dynamic weights to adjust the impact of different LoRAs. Experiments across six generative tasks demonstrate that our method consistently outperforms baselines with task-level fusion weights.
  arXiv  Detail & Related papers  (2024-02-18T04:41:25Z)
• Chain of LoRA: Efficient Fine-tuning of Language Models via Residual Learning [31.036465632204663]
  We introduce Chain of LoRA, an iterative optimization framework inspired by the Frank-Wolfe algorithm. We demonstrate that COLA can consistently outperform LoRA without additional computational or memory costs.
  arXiv  Detail & Related papers  (2024-01-08T14:26:49Z)
• SiRA: Sparse Mixture of Low Rank Adaptation [63.926732717719354]
  We investigate the importance of leveraging "sparse" computation and propose SiRA: sparse mixture of low rank. Specifically it enforces the top $k$ experts routing with a capacity limit restricting the maximum number of tokens each expert can process.
  arXiv  Detail & Related papers  (2023-11-15T18:15:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.