CSULoRA: Closest Safe Update Low-Rank Adaptation

• URL: http://arxiv.org/abs/2605.30640v1
• Date: Thu, 28 May 2026 22:48:54 GMT
• Title: CSULoRA: Closest Safe Update Low-Rank Adaptation
• Abstract summary: CSULoRA is a post-hoc method for correcting trained LoRA adapters through closest safe update estimation.<n>It solves a closed-form penalized minimum-change problem that preserves the fully aligned component while smoothly attenuating potentially unsafe directions according to their relative energy.<n>In adversarial fine-tuning experiments, CSULoRA substantially reduces attack success rate while preserving most of the utility gains obtained from standard LoRA fine-tuning.
• Score: 13.923849468280425
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Low-rank adaptation has become a standard method for parameter-efficient fine-tuning of large language models, but even small amounts of unsafe or adversarial fine-tuning data can substantially weaken the safety behavior of aligned models. Existing safety-preserving LoRA methods often rely on hard interventions such as projection, pruning, thresholding, or additional training objectives. While these methods can suppress unsafe update directions, they may also remove task-relevant information or require extra tuning. We introduce CSULoRA, a post-hoc method for correcting trained LoRA adapters through closest safe update estimation. CSULoRA estimates a safety-aligned subspace from the weight displacement between a safety-aligned model and its corresponding base checkpoint. It then decomposes each LoRA update into fully aligned, partially aligned, and off-subspace components. Instead of discarding components outside the estimated safety subspace, CSULoRA solves a closed-form penalized minimum-change problem that preserves the fully aligned component while smoothly attenuating potentially unsafe directions according to their relative energy. In adversarial fine-tuning experiments, CSULoRA substantially reduces attack success rate while preserving most of the utility gains obtained from standard LoRA fine-tuning.

Related papers

• Beyond SGD, Without SVD: Proximal Subspace Iteration LoRA with Diagonal Fractional K-FAC [50.36542772932594]
  Low-Rank Adaptation (LoRA) fine-tunes large models by learning low-rank updates on top of frozen weights.<n>In this work, we address the gap between training with full steps with low-rank projections (SVDLoRA) and LoRA fine-tuning.<n>We propose LoRSum, a memory-efficient subroutine that closes this gap for gradient descent.
  arXiv  Detail & Related papers  (2026-02-18T13:41:41Z)
• Safety Alignment as Continual Learning: Mitigating the Alignment Tax via Orthogonal Gradient Projection [52.551864761088574]
  Large Language Models (LLMs) often incur an alignment tax: safety post-training can reduce general utility.<n>We argue that this tax primarily arises from continual-learning-style forgetting in sequential alignment.<n>We propose Orthogonal Gradient Projection for Safety Alignment (OGPSA) to balance plasticity and stability.
  arXiv  Detail & Related papers  (2026-02-08T09:53:46Z)
• A Guardrail for Safety Preservation: When Safety-Sensitive Subspace Meets Harmful-Resistant Null-Space [91.99501941169831]
  GuardSpace is a guardrail framework for preserving safety alignment throughout fine-tuning.<n>For Llama-2-7B-Chat fine-tuned on GSM8K, GuardSpace outperforms the state-of-the-art method AsFT.
  arXiv  Detail & Related papers  (2025-10-16T04:57:53Z)
• Turning the Spell Around: Lightweight Alignment Amplification via Rank-One Safety Injection [47.347413305965006]
  Safety alignment in Large Language Models (LLMs) often involves mediating internal representations to refuse harmful requests.<n>Recent research has demonstrated that these safety mechanisms can be bypassed by ablating or removing specific representational directions.<n>We propose Rank-One Safety Injection (ROSI), a white-box method that amplifies a model's safety alignment by permanently steering its activations toward the refusal-mediating subspace.
  arXiv  Detail & Related papers  (2025-08-28T13:22:33Z)
• S3LoRA: Safe Spectral Sharpness-Guided Pruning in Adaptation of Agent Planner [1.6114012813668932]
  Adapting Large Language Models (LLMs) using parameter-efficient fine-tuning techniques such as LoRA has enabled powerful capabilities in LLM-based agents.<n>Existing safety-aware adaptation methods often require access to both base and instruction-tuned model checkpoints.<n>We propose S3LoRA, a lightweight, data-free, and model-independent framework that mitigates safety risks in LoRA-adapted models by inspecting only the fine-tuned weight updates.
  arXiv  Detail & Related papers  (2025-08-20T21:08:29Z)
• AlignGuard-LoRA: Alignment-Preserving Fine-Tuning via Fisher-Guided Decomposition and Riemannian-Geodesic Collision Regularization [6.5225344327304535]
  Low-rank adaptation (LoRA) has become a standard tool for efficiently fine-tuning large language models.<n>LoRA updates can induce alignment drift, weakening safety and behavioral constraints.<n>We propose AlignGuard-LoRA, a principled framework for preserving alignment during finetuning.
  arXiv  Detail & Related papers  (2025-08-04T05:45:24Z)
• LoRA is All You Need for Safety Alignment of Reasoning LLMs [30.616817385956754]
  We show that using LoRA for SFT on refusal datasets effectively aligns the model for safety without harming its reasoning capabilities.<n>This is because restricting the safety weight updates to a low-rank space minimizes the interference with the reasoning weights.<n>Our experiments show that this approach produces highly safe LLMs--with safety levels comparable to full-model fine-tuning--without compromising their reasoning abilities.
  arXiv  Detail & Related papers  (2025-07-22T23:25:16Z)
• Safe Pruning LoRA: Robust Distance-Guided Pruning for Safety Alignment in Adaptation of LLMs [4.580092836731863]
  Fine-tuning Large Language Models (LLMs) with Low-Rank Adaptation (LoRA) enhances adaptability while reducing computational costs.<n>Existing safety alignment methods struggle to capture complex parameter shifts, leading to suboptimal safety-utility trade-offs.<n>We propose Safe Pruning LoRA (SPLoRA), a novel pruning-based approach that selectively removes LoRA layers that weaken safety alignment.
  arXiv  Detail & Related papers  (2025-06-21T14:59:54Z)
• Shape it Up! Restoring LLM Safety during Finetuning [65.75757313781104]
  Finetuning large language models (LLMs) enables user-specific customization but introduces critical safety risks.<n>We propose dynamic safety shaping (DSS), a framework that uses fine-grained safety signals to reinforce learning from safe segments of a response while suppressing unsafe content.<n>We present STAR-DSS, guided by STAR scores, that robustly mitigates finetuning risks and delivers substantial safety improvements across diverse threats, datasets, and model families.
  arXiv  Detail & Related papers  (2025-05-22T18:05:16Z)
• SaLoRA: Safety-Alignment Preserved Low-Rank Adaptation [41.91948079316541]
  Recent studies have raised concerns that LoRA fine-tuning could potentially compromise the safety alignment in large language models.<n>In this paper, we propose Safety-alignment preserved Low-Rank Adaptation (SaLoRA)<n>Unlike previous LoRA methods and their variants, SaLoRA enables targeted modifications to LLMs without disrupting their original alignments.<n>Our experiments show that SaLoRA outperforms various adapters-based approaches across various evaluation metrics in different fine-tuning tasks.
  arXiv  Detail & Related papers  (2025-01-03T11:34:28Z)
• Safe LoRA: the Silver Lining of Reducing Safety Risks when Fine-tuning Large Language Models [51.20476412037321]
  We propose Safe LoRA, a simple one-liner patch to the original LoRA implementation by introducing the projection of LoRA weights from selected layers to the safety-aligned subspace.<n>Our experiments demonstrate that when fine-tuning on purely malicious data, Safe LoRA retains similar safety performance as the original aligned model.
  arXiv  Detail & Related papers  (2024-05-27T05:04:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.