Token-Level Adaptation of LoRA Adapters for Downstream Task Generalization

• URL: http://arxiv.org/abs/2311.10847v2
• Date: Fri, 1 Dec 2023 06:38:18 GMT
• Title: Token-Level Adaptation of LoRA Adapters for Downstream Task Generalization
• Abstract summary: This paper introduces a method for adapting LoRA adapters in smaller-sized language models to arbitrary downstream tasks. Our method employs a gradient-free routing function to choose a weighted combination of experts without increasing the compute requirements for training or inference.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper introduces a method for adapting LoRA adapters in smaller-sized language models to arbitrary downstream tasks. Unlike standard mixture-of-expert architectures, our method employs a gradient-free routing function to choose a weighted combination of experts without increasing the compute requirements for training or inference. The results show that token-level adaptation of LoRA adapters outperforms the base Llama-2-7b model across mathematical (GSM8K), scientific (ARC-Challenge), reading comprehension (SQuAD), and coding (CodeAlpaca-20k) tasks. Further evaluations also show that the average performance of token-level adaptation outperforms individual models fine-tuned for each of the tasks with the best performance observed in adaptation of every-other token during inference. The code for this study is made available through a public repository.

Related papers

• Automated Instruction Revision (AIR): A Structured Comparison of Task Adaptation Strategies for LLM [0.0]
  This paper studies Automated Instruction Revision (AIR), a rule-induction-based method for adapting large language models to downstream tasks.<n>We position AIR within the broader landscape of adaptation strategies, including prompt optimization, retrieval-based methods, and fine-tuning.<n>We compare these approaches across a diverse benchmark suite designed to stress different task requirements, such as knowledge injection, structured extraction, label remapping, and logical reasoning.
  arXiv  Detail & Related papers  (2026-04-10T15:28:05Z)
• Learning Adaptive LLM Decoding [6.643962667713069]
  We learn adaptive decoding policies that dynamically select sampling strategies at inference time, conditioned on available compute resources.<n>We introduce lightweight decoding adapters trained with reinforcement learning and verifiable terminal rewards.<n>Experiments show that the learned adapters improve the accuracy-budget tradeoff.
  arXiv  Detail & Related papers  (2026-03-10T01:15:26Z)
• Integrating Task-Specific and Universal Adapters for Pre-Trained Model-based Class-Incremental Learning [33.57130798344366]
  We propose integrating Task-Specific and Universal Adapters (TUNA) in this paper.<n> Specifically, we train task-specific adapters to capture the most crucial features relevant to their respective tasks.<n>We leverage an adapter fusion strategy to construct a universal adapter, which encodes the most discriminative features shared across tasks.
  arXiv  Detail & Related papers  (2025-08-11T16:41:04Z)
• Dynamic Context-oriented Decomposition for Task-aware Low-rank Adaptation with Less Forgetting and Faster Convergence [131.41894248194995]
  We propose context-oriented decomposition adaptation (CorDA), a novel method that initializes adapters in a task-aware manner.<n>Thanks to the task awareness, our method enables two optional adaptation modes, knowledge-preserved mode (KPM) and instruction-previewed mode (IPM)
  arXiv  Detail & Related papers  (2025-06-16T07:55:14Z)
• Taming LLMs by Scaling Learning Rates with Gradient Grouping [49.91587150497186]
  Training large language models (LLMs) poses challenges due to their massive scale and heterogeneous architectures.<n>This work introduces Scaling with Gradient Grouping (SGG), an gradient wrapper that improves adaptive learning rate estimation by dynamic grouping and group-specific scaling.
  arXiv  Detail & Related papers  (2025-06-01T15:30:37Z)
• Enhancing Training Data Attribution with Representational Optimization [57.61977909113113]
  Training data attribution methods aim to measure how training data impacts a model's predictions.<n>We propose AirRep, a representation-based approach that closes this gap by learning task-specific and model-aligned representations explicitly for TDA.<n>AirRep introduces two key innovations: a trainable encoder tuned for attribution quality, and an attention-based pooling mechanism that enables accurate estimation of group-wise influence.
  arXiv  Detail & Related papers  (2025-05-24T05:17:53Z)
• Enhancing Learned Knowledge in LoRA Adapters Through Efficient Contrastive Decoding on Ascend NPUs [12.995753143157083]
  Contrastive LoRA Decoding (CoLD) is a novel decoding framework designed to maximize the use of task-specific knowledge in LoRA-adapted models.<n>CoLD achieves up to a 5.54% increase in task accuracy while reducing end-to-end latency by 28%.
  arXiv  Detail & Related papers  (2025-05-20T17:11:18Z)
• Semantic Library Adaptation: LoRA Retrieval and Fusion for Open-Vocabulary Semantic Segmentation [72.28364940168092]
  Open-vocabulary semantic segmentation models associate vision and text to label pixels from an undefined set of classes using textual queries. We introduce Semantic Library Adaptation (SemLA), a novel framework for training-free, test-time domain adaptation.
  arXiv  Detail & Related papers  (2025-03-27T17:59:58Z)
• Rank Also Matters: Hierarchical Configuration for Mixture of Adapter Experts in LLM Fine-Tuning [5.074620301447097]
  We propose a hierarchical scheme for expert allocation and rank configuration, HILO, for large language models (LLMs) HILO dynamically adjusts the number and rank of adapter experts across layers, matching the varying representational complexity of model layers in adapter-granularity. Experiments on multiple benchmark tasks demonstrate that HILO outperforms existing methods in accuracy while introducing fewer trainable parameters.
  arXiv  Detail & Related papers  (2025-02-06T08:58:03Z)
• Ensembles of Low-Rank Expert Adapters [9.599957499802446]
  We propose the Ensembles of Low-Rank Expert Adapters (ELREA) framework to improve the model's capability to handle diverse tasks. ELREA clusters the training instructions based on their gradient directions, representing different areas of expertise. During inference, ELREA combines predictions from the most relevant expert adapters based on the input data's gradient similarity to the training clusters.
  arXiv  Detail & Related papers  (2025-01-31T18:07:21Z)
• Less is More: Extreme Gradient Boost Rank-1 Adaption for Efficient Finetuning of LLMs [75.11449420928139]
  Fine-tuning Large Language Models (LLMs) has become a crucial technique for adapting pre-trained models to downstream tasks. Low-Rank Adaptation (LoRA) has emerged as a promising solution, but there exists a gap between the practical performance of low-rank adaptations and its theoretical optimum. We propose eXtreme Gradient Boosting LoRA, a novel framework that bridges this gap by leveraging the power of ensemble learning.
  arXiv  Detail & Related papers  (2024-10-25T17:07:13Z)
• Collaborative and Efficient Personalization with Mixtures of Adaptors [5.195669033269619]
  We propose a parameter-efficient framework to tackle multi-task learning problems. We call our framework Federated Low-Rank Adaptive Learning (FLoRAL) We show promising experimental results on synthetic datasets and real-world federated multi-task problems.
  arXiv  Detail & Related papers  (2024-10-04T15:11:15Z)
• Adaptive Adapter Routing for Long-Tailed Class-Incremental Learning [55.384428765798496]
  New data exhibits a long-tailed distribution, such as e-commerce platform reviews. This necessitates continuous model learning imbalanced data without forgetting. We introduce AdaPtive Adapter RouTing (APART) as an exemplar-free solution for LTCIL.
  arXiv  Detail & Related papers  (2024-09-11T17:52:00Z)
• Towards Modular LLMs by Building and Reusing a Library of LoRAs [64.43376695346538]
  We study how to best build a library of adapters given multi-task data. We introduce model-based clustering, MBC, a method that groups tasks based on the similarity of their adapter parameters. To re-use the library, we present a novel zero-shot routing mechanism, Arrow, which enables dynamic selection of the most relevant adapters.
  arXiv  Detail & Related papers  (2024-05-18T03:02:23Z)
• The Languini Kitchen: Enabling Language Modelling Research at Different Scales of Compute [66.84421705029624]
  We introduce an experimental protocol that enables model comparisons based on equivalent compute, measured in accelerator hours. We pre-process an existing large, diverse, and high-quality dataset of books that surpasses existing academic benchmarks in quality, diversity, and document length. This work also provides two baseline models: a feed-forward model derived from the GPT-2 architecture and a recurrent model in the form of a novel LSTM with ten-fold throughput.
  arXiv  Detail & Related papers  (2023-09-20T10:31:17Z)
• One-for-All: Generalized LoRA for Parameter-Efficient Fine-tuning [34.109808214968176]
  Generalized LoRA (GLoRA) is an advanced approach for universal parameter-efficient fine-tuning tasks. It employs a generalized prompt module to optimize pre-trained model weights and adjust intermediate activations. GLoRA exhibits strong transfer learning, few-shot learning and domain generalization abilities.
  arXiv  Detail & Related papers  (2023-06-13T17:59:32Z)
• OWAdapt: An adaptive loss function for deep learning using OWA operators [0.8437187555622164]
  We propose a fuzzy adaptive loss function for enhancing deep learning performance in classification tasks. Our approach introduces aggregation operators, leveraging the power of fuzzy logic to improve classification accuracy. Our method outperforms other commonly used loss functions, such as the standard cross-entropy or focal loss.
  arXiv  Detail & Related papers  (2023-05-30T22:34:48Z)
• Retrieval as Attention: End-to-end Learning of Retrieval and Reading within a Single Transformer [80.50327229467993]
  We show that a single model trained end-to-end can achieve both competitive retrieval and QA performance. We show that end-to-end adaptation significantly boosts its performance on out-of-domain datasets in both supervised and unsupervised settings.
  arXiv  Detail & Related papers  (2022-12-05T04:51:21Z)
• Contextual Squeeze-and-Excitation for Efficient Few-Shot Image Classification [57.36281142038042]
  We present a new adaptive block called Contextual Squeeze-and-Excitation (CaSE) that adjusts a pretrained neural network on a new task to significantly improve performance. We also present a new training protocol based on Coordinate-Descent called UpperCaSE that exploits meta-trained CaSE blocks and fine-tuning routines for efficient adaptation.
  arXiv  Detail & Related papers  (2022-06-20T15:25:08Z)
• Adaptive Sampling for Minimax Fair Classification [40.936345085421955]
  We propose an adaptive sampling algorithm based on the principle of optimism, and derive theoretical bounds on its performance. By deriving algorithm independent lower-bounds for a specific class of problems, we show that the performance achieved by our adaptive scheme cannot be improved in general.
  arXiv  Detail & Related papers  (2021-03-01T04:58:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.