Foundation-Preserving Adaptation via Generalized Rayleigh-Quotient Optimization

• URL: http://arxiv.org/abs/2606.00132v1
• Date: Thu, 28 May 2026 21:22:31 GMT
• Title: Foundation-Preserving Adaptation via Generalized Rayleigh-Quotient Optimization
• Abstract summary: We propose Foundation Preserving LoRA (FoLoRA), a forgetting aware optimization framework.<n>FoLoRA defines a forgetting penalty over pretraining-proxy activations and a task utility over downstream task activations.<n>Experiments on math, code, and instruction following adaptation show that FoLoRA achieves the strongest preservation adaptation balance over baselines.
• Score: 36.95831224411691
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While finetuning effectively adapts foundation models to specialized downstream tasks, it can degrade nontarget capabilities acquired during pretraining. Existing forgetting aware methods typically seek safer updates through specialized initialization or fixed constraints, but do not regulate the adaptation preservation trade-off during training. We propose Foundation Preserving LoRA (FoLoRA), a forgetting aware optimization framework. Guided by a first order preservation condition, FoLoRA defines a forgetting penalty over pretraining-proxy activations and a task utility over downstream task activations. It then scores update directions by task utility per unit forgetting penalty via a generalized Rayleigh quotient. The resulting spectral coordinate system enables direction wise gated Adam updates, attenuating low utility to penalty directions during training. To estimate the forgetting penalty, FoLoRA constructs pretraining proxy calibration data by sampling from the pretrained model rather than relying on a single proxy dataset. Experiments on math, code, and instruction following adaptation show that FoLoRA achieves the strongest preservation adaptation balance over baselines, improving target task performance with best aggregate preservation of non target capabilities.

Related papers

• ADORN: Adaptive Drift handling for Open RAN using Reinforcement Learning [0.9577207941409713]
  Dynamic traffic variations in Open Radio Access Networks (O-RAN) lead to drift.<n>Traditional retraining approaches maintain forecasting accuracy but incur high computational cost.<n>This work proposes a Q-learning-based adaptive retraining approach that formulates the retraining decision as a Markov Decision Process.
  arXiv  Detail & Related papers  (2026-07-09T13:05:42Z)
• Mask the Target: A Plug-and-Play Regularizer Against LoRA Forgetting [28.32780832670352]
  Low-Rank Adaptation (LoRA) has become one of the most widely used fine-tuning mechanisms for adapting large language models.<n>We show that this forgetting becomes especially severe when the adaptation distribution differs substantially from the models original training or alignment distributions.<n>Motivated by this constraint, we study how LoRA based adaptation balances new learning against forgetting in a replay-free setting.
  arXiv  Detail & Related papers  (2026-05-28T07:22:39Z)
• Rule-based High-Level Coaching for Goal-Conditioned Reinforcement Learning in Search-and-Rescue UAV Missions Under Limited-Simulation Training [1.1960178399478718]
  The framework combines a fixed rule-based high-level advisor with an online goal-conditioned low-level reinforcement learning controller.<n>We evaluate the framework on two tasks: battery-aware multi-goal delivery and moving-target delivery in obstacle-rich environments.
  arXiv  Detail & Related papers  (2026-04-29T16:01:22Z)
• RFS: Reinforcement Learning with Residual Flow Steering for Dexterous Manipulation [7.500999283386335]
  Residual Flow Steering (RFS) is a data-efficient reinforcement learning framework for adapting pretrained generative policies.<n>RFS steers a pretrained flow-matching policy by jointly optimizing a residual action and a latent noise distribution.<n>We demonstrate the effectiveness of RFS on dexterous manipulation tasks, showing efficient fine-tuning in both simulation and real-world settings.
  arXiv  Detail & Related papers  (2026-02-02T08:11:57Z)
• Stabilizing Reinforcement Learning with LLMs: Formulation and Practices [61.361819972410046]
  We show why and under what conditions the true sequence-level reward can be optimized via a surrogate token-level objective in policy gradient methods such as REINFORCE.<n>This insight provides a principled explanation for the crucial role of several widely adopted techniques in stabilizing RL training.
  arXiv  Detail & Related papers  (2025-12-01T07:45:39Z)
• Fast Adaptation with Behavioral Foundation Models [82.34700481726951]
  Unsupervised zero-shot reinforcement learning has emerged as a powerful paradigm for pretraining behavioral foundation models.<n>Despite promising results, zero-shot policies are often suboptimal due to errors induced by the unsupervised training process.<n>We propose fast adaptation strategies that search in the low-dimensional task-embedding space of the pre-trained BFM to rapidly improve the performance of its zero-shot policies.
  arXiv  Detail & Related papers  (2025-04-10T16:14:17Z)
• TapWeight: Reweighting Pretraining Objectives for Task-Adaptive Pretraining [34.93043212352875]
  TapWeight is a task-adaptive pretraining framework which automatically determines the optimal importance of each pretraining objective. We applied TapWeight to both molecular property prediction and natural language understanding tasks, significantly surpassing baseline methods.
  arXiv  Detail & Related papers  (2024-10-13T20:56:13Z)
• Enhancing Robustness of Vision-Language Models through Orthogonality Learning and Self-Regularization [77.62516752323207]
  We introduce an orthogonal fine-tuning method for efficiently fine-tuning pretrained weights and enabling enhanced robustness and generalization. A self-regularization strategy is further exploited to maintain the stability in terms of zero-shot generalization of VLMs, dubbed OrthSR. For the first time, we revisit the CLIP and CoOp with our method to effectively improve the model on few-shot image classficiation scenario.
  arXiv  Detail & Related papers  (2024-07-11T10:35:53Z)
• Stable Distillation: Regularizing Continued Pre-training for Low-Resource Automatic Speech Recognition [54.9235160379917]
  Stable Distillation is a simple and novel approach for SSL-based continued pre-training. It boosts ASR performance in the target domain where both labeled and unlabeled data are limited.
  arXiv  Detail & Related papers  (2023-12-20T06:02:12Z)
• Sparse Low-rank Adaptation of Pre-trained Language Models [79.74094517030035]
  We introduce sparse low-rank adaptation (SoRA) that enables dynamic adjustments to the intrinsic rank during the adaptation process. Our approach strengthens the representation power of LoRA by initializing it with a higher rank, while efficiently taming a temporarily increased number of parameters. Our experimental results demonstrate that SoRA can outperform other baselines even with 70% retained parameters and 70% training time.
  arXiv  Detail & Related papers  (2023-11-20T11:56:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.