Mask the Target: A Plug-and-Play Regularizer Against LoRA Forgetting

• URL: http://arxiv.org/abs/2605.29498v1
• Date: Thu, 28 May 2026 07:22:39 GMT
• Title: Mask the Target: A Plug-and-Play Regularizer Against LoRA Forgetting
• Abstract summary: Low-Rank Adaptation (LoRA) has become one of the most widely used fine-tuning mechanisms for adapting large language models.<n>We show that this forgetting becomes especially severe when the adaptation distribution differs substantially from the models original training or alignment distributions.<n>Motivated by this constraint, we study how LoRA based adaptation balances new learning against forgetting in a replay-free setting.
• Score: 28.32780832670352
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Low-Rank Adaptation (LoRA) has become one of the most widely used fine-tuning mechanisms for adapting large language models to new domains, tasks, and users. Yet adaptation performance alone can obscure an important failure mode: LoRA updates may improve performance on the target distribution while degrading prior capabilities learned during pretraining and alignment. We show that this forgetting becomes especially severe when the adaptation distribution differs substantially from the models original training or alignment distributions. The challenge is amplified in practical settings, where the original training and alignment data are typically unavailable. Motivated by this constraint, we study how LoRA based adaptation balances new learning against forgetting in a replay-free setting, and introduce a simple output space regularizer that can be added directly to existing training pipelines. Our method removes the ground-truth token from both the base and adapted model distributions, renormalizes the remaining probabilities, and applies KL regularization only over the non-target vocabulary. This preserves the base models relative preferences among alternative tokens without directly opposing the cross-entropy signal required for adaptation. As the regularizer acts only at the loss level, it requires no replay data, architectural changes, adapter redesign, or inference-time overhead, and can be applied directly to existing LoRA variants. Across all LoRA variants tested and across various backbones, our method improves the frontier between new learning and forgetting when the adaptation distribution differs substantially from the base models original training or alignment distributions, suggesting a broadly applicable route toward more reliable LLM updating.

Related papers

• Beyond LoRA: Is Sparsity-Induced Adaptation Better? [10.18127129139535]
  Low-rank adaptation (LoRA) and its variants provide a memory-matched and compute-efficient alternative to full fine-tuning of pre-trained models.<n>We present a historical framing, covering the past (full fine-tuning and original LoRA), the present (different variants of LoRA) and propose simpler, cheaper, parameter-efficient extensions.<n>We evaluate 11 fine-tuning methods across 10 pre-trained models and 14 datasets, analyzing the fine-tuned models' performance and generalization using tools such as loss landscapes and spectral analysis.
  arXiv  Detail & Related papers  (2026-06-11T17:59:01Z)
• CRAFT: Forgetting-Aware Intervention-Based Adaptation for Continual Learning [7.180013165859104]
  CRAFT is a continual learning framework that avoids updating model weights.<n>It first routes each task to a group of similar tasks based on output-distribution divergence.<n>It then fine-tunes the model using a Kullback-Leibler (KL) divergence against the group's prior state.<n>Finally, it merges interventions for the updated task into the shared representation using the same KL signal.
  arXiv  Detail & Related papers  (2026-05-07T06:24:13Z)
• LoFA: Learning to Predict Personalized Priors for Fast Adaptation of Visual Generative Models [50.46815266062554]
  Low-Rank Adaptation (LoRA) remains impractical due to their demand for task-specific data and lengthy optimization.<n>We propose LoFA, a general framework that efficiently predicts personalized priors for fast model adaptation.<n>Our method consistently predicts high-quality personalized priors within seconds, across multiple tasks and user prompts, even outperforming conventional LoRA that requires hours of processing.
  arXiv  Detail & Related papers  (2025-12-09T16:39:31Z)
• Test Time Adaptation Using Adaptive Quantile Recalibration [19.97106215064574]
  Domain adaptation is a key strategy for enhancing the generalizability of deep learning models in real-world scenarios.<n>Recent test-time adaptation methods based on batch normalization statistic updates allow for unsupervised adaptation.<n>We propose Adaptive Quantile Recalibration (AQR), a test-time adaptation technique that modifies pre-activation distributions by aligning quantiles on a channel-wise basis.
  arXiv  Detail & Related papers  (2025-11-05T03:12:30Z)
• Regularizing Subspace Redundancy of Low-Rank Adaptation [54.473090597164834]
  We propose ReSoRA, a method that explicitly models redundancy between mapping subspaces and adaptively Regularizes Subspace redundancy of Low-Rank Adaptation.<n>Our proposed method consistently facilitates existing state-of-the-art PETL methods across various backbones and datasets in vision-language retrieval and standard visual classification benchmarks.<n>As a training supervision, ReSoRA can be seamlessly integrated into existing approaches in a plug-and-play manner, with no additional inference costs.
  arXiv  Detail & Related papers  (2025-07-28T11:52:56Z)
• SD-LoRA: Scalable Decoupled Low-Rank Adaptation for Class Incremental Learning [73.93639228235622]
  Continual Learning with foundation models has emerged as a promising paradigm to exploit abundant knowledge acquired during pre-training for tackling sequential tasks.<n>Existing prompt-based and Low-Rank Adaptation-based (LoRA-based) methods often require expanding a prompt/LoRA pool or retaining samples of previous tasks.<n>We propose Scalable Decoupled LoRA (SD-LoRA) for class incremental learning, which continually separates the learning of the magnitude and direction of LoRA components without rehearsal.
  arXiv  Detail & Related papers  (2025-01-22T20:00:41Z)
• Unlocking Tuning-Free Few-Shot Adaptability in Visual Foundation Models by Recycling Pre-Tuned LoRAs [76.40876036912537]
  Large Language Models (LLMs) demonstrate strong few-shot adaptability without requiring fine-tuning.<n>Current Visual Foundation Models (VFMs) require explicit fine-tuning with sufficient tuning data.<n>We propose a framework, LoRA Recycle, that distills a meta-LoRA from diverse pre-tuned LoRAs with a meta-learning objective.
  arXiv  Detail & Related papers  (2024-12-03T07:25:30Z)
• Training Language Models to Self-Correct via Reinforcement Learning [98.35197671595343]
  Self-correction has been found to be largely ineffective in modern large language models (LLMs) We develop a multi-turn online reinforcement learning approach, SCoRe, that significantly improves an LLM's self-correction ability using entirely self-generated data. We find that SCoRe achieves state-of-the-art self-correction performance, improving the base models' self-correction by 15.6% and 9.1% respectively on MATH and HumanEval.
  arXiv  Detail & Related papers  (2024-09-19T17:16:21Z)
• Sparse Low-rank Adaptation of Pre-trained Language Models [79.74094517030035]
  We introduce sparse low-rank adaptation (SoRA) that enables dynamic adjustments to the intrinsic rank during the adaptation process. Our approach strengthens the representation power of LoRA by initializing it with a higher rank, while efficiently taming a temporarily increased number of parameters. Our experimental results demonstrate that SoRA can outperform other baselines even with 70% retained parameters and 70% training time.
  arXiv  Detail & Related papers  (2023-11-20T11:56:25Z)
• Learning Invariant Representations and Risks for Semi-supervised Domain Adaptation [109.73983088432364]
  We propose the first method that aims to simultaneously learn invariant representations and risks under the setting of semi-supervised domain adaptation (Semi-DA) We introduce the LIRR algorithm for jointly textbfLearning textbfInvariant textbfRepresentations and textbfRisks.
  arXiv  Detail & Related papers  (2020-10-09T15:42:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.