Activated LoRA: Fine-tuned LLMs for Intrinsics

• URL: http://arxiv.org/abs/2504.12397v2
• Date: Tue, 29 Apr 2025 14:25:08 GMT
• Title: Activated LoRA: Fine-tuned LLMs for Intrinsics
• Authors: Kristjan Greenewald, Luis Lastras, Thomas Parnell, Vraj Shah, Lucian Popa, Giulio Zizzo, Chulaka Gunasekara, Ambrish Rawat, David Cox,
• Abstract summary: Low-Rank Adaptation (LoRA) has emerged as a highly efficient framework for finetuning the weights of large foundation models.<n>We propose Activated LoRA (aLoRA), which modifies the LoRA framework to only adapt weights for the tokens in the sequence emphafter the aLoRA is invoked.<n>This change crucially allows aLoRA to accept the base model's KV cache of the input string, meaning that aLoRA can be instantly activated whenever needed in a chain.
• Score: 9.503174205896533
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Low-Rank Adaptation (LoRA) has emerged as a highly efficient framework for finetuning the weights of large foundation models, and has become the go-to method for data-driven customization of LLMs. Despite the promise of highly customized behaviors and capabilities, switching between relevant LoRAs in a multiturn setting is highly inefficient, as the key-value (KV) cache of the entire turn history must be recomputed with the LoRA weights before generation can begin. To address this problem, we propose Activated LoRA (aLoRA), which modifies the LoRA framework to only adapt weights for the tokens in the sequence \emph{after} the aLoRA is invoked. This change crucially allows aLoRA to accept the base model's KV cache of the input string, meaning that aLoRA can be instantly activated whenever needed in a chain without recomputing the cache. This enables building what we call \emph{intrinsics}, i.e. highly specialized models invoked to perform well-defined operations on portions of an input chain or conversation that otherwise uses the base model by default. We use aLoRA to train a set of intrinsics models, demonstrating competitive accuracy with standard LoRA while achieving significant inference benefits.

Related papers

• LoRA Done RITE: Robust Invariant Transformation Equilibration for LoRA Optimization [78.93425154518705]
  Low-rank adaption (LoRA) is a widely used parameter-efficient finetuning method for LLM that reduces memory requirements. This paper introduces LoRA-RITE, a novel adaptive matrix preconditioning method for LoRA optimization.
  arXiv  Detail & Related papers  (2024-10-27T22:57:12Z)
• NoRA: Nested Low-Rank Adaptation for Efficient Fine-Tuning Large Models [27.757883818520217]
  Nested Low-Rank Adaptation (NoRA) is a novel approach to parameter-efficient fine-tuning. By freezing outer LoRA weights and using an inner LoRA design, NoRA enables precise task adaptation with a compact parameter space.
  arXiv  Detail & Related papers  (2024-08-18T12:18:56Z)
• LoRA-XS: Low-Rank Adaptation with Extremely Small Number of Parameters [11.23006032094776]
  We introduce LoRA-XS, a novel low-rank adaptation method that considerably reduces the trainable parameters while showing superior or competitive performance. LoRA-XS achieves a remarkable reduction of trainable parameters by over 100x in 7B models compared to LoRA.
  arXiv  Detail & Related papers  (2024-05-27T19:07:13Z)
• Safe LoRA: the Silver Lining of Reducing Safety Risks when Fine-tuning Large Language Models [51.20476412037321]
  We propose Safe LoRA, a simple one-liner patch to the original LoRA implementation by introducing the projection of LoRA weights from selected layers to the safety-aligned subspace. Our experiments demonstrate that when fine-tuning on purely malicious data, Safe LoRA retains similar safety performance as the original aligned model.
  arXiv  Detail & Related papers  (2024-05-27T05:04:05Z)
• ResLoRA: Identity Residual Mapping in Low-Rank Adaption [96.59370314485074]
  We propose ResLoRA, an improved framework of low-rank adaptation (LoRA) Our method can achieve better results in fewer training steps without any extra trainable parameters or inference cost compared to LoRA. The experiments on NLG, NLU, and text-to-image tasks demonstrate the effectiveness of our method.
  arXiv  Detail & Related papers  (2024-02-28T04:33:20Z)
• LoRA-Flow: Dynamic LoRA Fusion for Large Language Models in Generative Tasks [72.88244322513039]
  LoRA employs lightweight modules to customize large language models (LLMs) for each downstream task or domain. We propose LoRA-Flow, which utilizes dynamic weights to adjust the impact of different LoRAs. Experiments across six generative tasks demonstrate that our method consistently outperforms baselines with task-level fusion weights.
  arXiv  Detail & Related papers  (2024-02-18T04:41:25Z)
• DoRA: Weight-Decomposed Low-Rank Adaptation [57.68678247436207]
  We introduce a novel weight decomposition analysis to investigate the inherent differences between FT and LoRA. Aiming to resemble the learning capacity of FT from the findings, we propose Weight-Decomposed Low-Rank Adaptation (DoRA) DoRA decomposes the pre-trained weight into two components, magnitude and direction, for fine-tuning.
  arXiv  Detail & Related papers  (2024-02-14T17:59:34Z)
• Chain of LoRA: Efficient Fine-tuning of Language Models via Residual Learning [31.036465632204663]
  We introduce Chain of LoRA, an iterative optimization framework inspired by the Frank-Wolfe algorithm. We demonstrate that COLA can consistently outperform LoRA without additional computational or memory costs.
  arXiv  Detail & Related papers  (2024-01-08T14:26:49Z)
• S-LoRA: Serving Thousands of Concurrent LoRA Adapters [59.490751234925206]
  Low-Rank Adaptation (LoRA), a parameter-efficient fine-tuning method, is often employed to adapt a base model to a multitude of tasks. We present S-LoRA, a system designed for the scalable serving of many LoRA adapters.
  arXiv  Detail & Related papers  (2023-11-06T17:26:17Z)
• LoRA-FA: Memory-efficient Low-rank Adaptation for Large Language Models Fine-tuning [19.08716369943138]
  We present LoRA-FA, a memory-efficient fine-tuning method that reduces the activation memory without performance degradation and expensive recomputation. Our results show that LoRA-FA can always achieve close fine-tuning accuracy across different tasks compared to full parameter fine-tuning and LoRA.
  arXiv  Detail & Related papers  (2023-08-07T05:12:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.