When Safety Routing Breaks: Understanding Alignment Fragility under Benign Fine-Tuning

• URL: http://arxiv.org/abs/2609.01455v1
• Date: Tue, 01 Sep 2026 15:59:32 GMT
• Title: When Safety Routing Breaks: Understanding Alignment Fragility under Benign Fine-Tuning
• Abstract summary: We show that safety Fisher is low-rank, and alignment makes the safety geometry flatter while preserving an output-routing pathway.<n>LoRA and ASAM mitigate early collapse by suppressing output-side sharpness, but their protection weakens at larger fine-tuning scales.<n>Overall, safety failure is best understood as a disruption of a low-rank output-routing mechanism.
• Score: 19.41242962849505
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Benign fine-tuning severely weakens the safety alignment of large language models (LLMs), so we study why refusal behavior is so fragile. While prior work often attributes this failure to gradient conflict, we propose a fundamentally different Fisher-geometric explanation: safety Fisher is low-rank, and alignment makes the safety geometry flatter while preserving an output-routing pathway. After 100 benign fine-tuning examples, this pathway is selectively re-sharpened in output-side MLP modules, explaining the asymmetric fragility: safety can collapse to high attack success rates, while general utility degrades mildly. The routing view also explains why few safety examples can restore refusal behavior, indicating that internal safety-relevant representations are preserved. Finally, we show that LoRA and ASAM mitigate early collapse by suppressing output-side sharpness, but their protection weakens at larger fine-tuning scales. Overall, safety failure is best understood as a disruption of a low-rank output-routing mechanism

Related papers

• When Autoregressive Consistency Hurts Safety Alignment [20.12397699480725]
  We show that autoregressive consistency can concentrate alignment updates on early tokens.<n>The same mechanism also predicts a broader class of attacks on large language models.<n>We propose adversarial safety alignment, an initial framework based on worst-case harmful continuation states.
  arXiv  Detail & Related papers  (2026-06-02T19:32:23Z)
• RefusalGuard: Geometry-Preserving Fine-Tuning for Safety in LLMs [5.100622189286672]
  Fine-tuning safety-aligned language models for downstream tasks can lead to substantial degradation of refusal behavior.<n>We introduce REFUSALGUARD, a representation-level fine-tuning framework that preserves safety-relevant structure during model adaptation.
  arXiv  Detail & Related papers  (2026-05-03T14:48:18Z)
• THINKSAFE: Self-Generated Safety Alignment for Reasoning Models [60.10077024249373]
  We propose ThinkSafe, a framework that restores safety alignment without external teachers.<n>Our key insight is that while compliance suppresses safety mechanisms, models often retain latent knowledge to identify harm.<n> Experiments on DeepSeek-R1-Distill and Qwen3 show ThinkSafe significantly improves safety while preserving reasoning proficiency.
  arXiv  Detail & Related papers  (2026-01-30T16:31:02Z)
• Safety at One Shot: Patching Fine-Tuned LLMs with A Single Instance [20.0828672005664]
  We show that safety alignment can be fully recovered with only a single safety example.<n>We uncover the low-rank structure of the safety gradient, which explains why such efficient correction is possible.
  arXiv  Detail & Related papers  (2026-01-05T08:26:34Z)
• Refusal Falls off a Cliff: How Safety Alignment Fails in Reasoning? [68.82210578851442]
  We investigate why safety alignment fails in reasoning models through a mechanistic interpretability lens.<n>Using a linear probing approach to trace refusal intentions across token positions, we discover a phenomenon termed as textbfrefusal cliff<n>We propose textbfCliff-as-a-Judge, a novel data selection method that identifies training examples exhibiting the largest refusal cliff to efficiently repair reasoning models' safety alignment.
  arXiv  Detail & Related papers  (2025-10-07T15:32:59Z)
• Turning the Spell Around: Lightweight Alignment Amplification via Rank-One Safety Injection [47.347413305965006]
  Safety alignment in Large Language Models (LLMs) often involves mediating internal representations to refuse harmful requests.<n>Recent research has demonstrated that these safety mechanisms can be bypassed by ablating or removing specific representational directions.<n>We propose Rank-One Safety Injection (ROSI), a white-box method that amplifies a model's safety alignment by permanently steering its activations toward the refusal-mediating subspace.
  arXiv  Detail & Related papers  (2025-08-28T13:22:33Z)
• Why LLM Safety Guardrails Collapse After Fine-tuning: A Similarity Analysis Between Alignment and Fine-tuning Datasets [64.96967819446553]
  This paper investigates the degradation of safety guardrails through the lens of representation similarity between upstream alignment datasets and downstream fine-tuning tasks.<n>High similarity between these datasets significantly weakens safety guardrails, making models more susceptible to jailbreaks.<n>Low similarity between these two types of datasets yields substantially more robust models and thus reduces harmfulness score by up to 10.33%.
  arXiv  Detail & Related papers  (2025-06-05T17:59:55Z)
• Shape it Up! Restoring LLM Safety during Finetuning [65.75757313781104]
  Finetuning large language models (LLMs) enables user-specific customization but introduces critical safety risks.<n>We propose dynamic safety shaping (DSS), a framework that uses fine-grained safety signals to reinforce learning from safe segments of a response while suppressing unsafe content.<n>We present STAR-DSS, guided by STAR scores, that robustly mitigates finetuning risks and delivers substantial safety improvements across diverse threats, datasets, and model families.
  arXiv  Detail & Related papers  (2025-05-22T18:05:16Z)
• Safety Alignment Should Be Made More Than Just a Few Tokens Deep [48.823599143711235]
  The safety alignment of current Large Language Models (LLMs) is vulnerable. Relatively simple attacks, or even benign fine-tuning, can jailbreak aligned models. We show that deepening the safety alignment beyond just the first few tokens can often meaningfully improve robustness against some common exploits.
  arXiv  Detail & Related papers  (2024-06-10T00:35:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.