Finding and Reactivating Post-Trained LLMs' Hidden Safety Mechanisms

• URL: http://arxiv.org/abs/2604.00012v1
• Date: Tue, 10 Mar 2026 06:56:23 GMT
• Title: Finding and Reactivating Post-Trained LLMs' Hidden Safety Mechanisms
• Abstract summary: This paper investigates the underlying cause of safety degradation in large language models (LLMs)<n>We propose a lightweight and cost-effective solution called SafeReAct that restores the suppressed safety behaviors by aligning with LoRA adapters.<n> Experiments on four state-of-the-art LRMs show that our method significantly improves safety on harmful prompts without compromising reasoning performance.
• Score: 43.21675006313594
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Despite the impressive performance of general-purpose large language models (LLMs), they often require fine-tuning or post-training to excel at specific tasks. For instance, large reasoning models (LRMs), such as the DeepSeek-R1 series, demonstrate strong reasoning capabilities after post-training different general large language models on diverse chain-of-thought (CoT) datasets. However, this additional training frequently comes at the cost of reduced safety, as the fine-tuned or post-trained models tend to exhibit more harmful behaviors compared with the regular LLMs before post-training or fine-tuning, potentially leading to harmful outcomes due to their enhanced capabilities. Taking LRMs as an example, we first investigate the underlying cause of this safety degradation in this paper. Our analysis reveals that post-training can mask the original safety mechanisms of the base LLM, while over-amplifying representations related to their post-training ability. But luckily, we also find that LRMs' safety mechanisms still exist instead of being removed during their post-training. Based on these findings, we propose a lightweight and cost-effective solution called SafeReAct that restores the suppressed safety behaviors by aligning with LoRA adapters on a few layers. Experiments on four state-of-the-art LRMs show that our method significantly improves safety on harmful prompts without compromising reasoning performance. Besides LRMs, additional results on other domain-specific LLMs, like medical models, further confirm the generality and effectiveness of our approach.

Related papers

• An Efficient and Modular Framework for Targeted Harm Mitigation in LLMS [3.37470725921294]
  Large Language Models (LLMs) are powerful zero-shot learners but remain prone to misalignment with human preferences.<n>We propose a modular correction framework that augments pretrained LLMs with Activated LoRA adapters and a context-aware routing mechanism.
  arXiv  Detail & Related papers  (2026-09-12T00:24:49Z)
• Beyond SFT: Reinforcement Learning for Safer Large Reasoning Models with Better Reasoning Ability [18.931331452604066]
  Large reasoning models (LRMs) extend large language models by generating explicit chain-of-thought (CoT) reasoning.<n>Existing safety alignment approaches rely on supervised fine-tuning (SFT) over safety-oriented long CoT datasets.<n>We investigate reinforcement learning (RL) as a complementary optimization framework for LRM safety training.
  arXiv  Detail & Related papers  (2025-12-01T16:35:34Z)
• When Models Outthink Their Safety: Mitigating Self-Jailbreak in Large Reasoning Models with Chain-of-Guardrails [74.63933201261595]
  Large Reasoning Models (LRMs) demonstrate remarkable capabilities on complex reasoning tasks.<n>LRMs remain vulnerable to severe safety risks, including harmful content generation and jailbreak attacks.<n>We propose the Chain-of-Guardrail (CoG), a training framework that recomposes or backtracks unsafe reasoning steps.
  arXiv  Detail & Related papers  (2025-10-24T09:32:25Z)
• R1-ACT: Efficient Reasoning Model Safety Alignment by Activating Safety Knowledge [16.653490433862224]
  We investigate the underlying cause of LRM safety risks and find that models already possess sufficient safety knowledge but fail to activate it during reasoning.<n>We propose R1-Act, a simple and efficient post-training method that explicitly triggers safety knowledge through a structured reasoning process.<n>R1-Act achieves strong safety improvements while preserving reasoning performance, outperforming prior alignment methods.
  arXiv  Detail & Related papers  (2025-08-01T05:14:13Z)
• Saffron-1: Safety Inference Scaling [69.61130284742353]
  SAFFRON is a novel inference scaling paradigm tailored explicitly for safety assurance.<n>Central to our approach is the introduction of a multifurcation reward model (MRM) that significantly reduces the required number of reward model evaluations.<n>We publicly release our trained multifurcation reward model (Saffron-1) and the accompanying token-level safety reward dataset (Safety4M)
  arXiv  Detail & Related papers  (2025-06-06T18:05:45Z)
• SafeMLRM: Demystifying Safety in Multi-modal Large Reasoning Models [50.34706204154244]
  Acquiring reasoning capabilities catastrophically degrades inherited safety alignment.<n>Certain scenarios suffer 25 times higher attack rates.<n>Despite tight reasoning-answer safety coupling, MLRMs demonstrate nascent self-correction.
  arXiv  Detail & Related papers  (2025-04-09T06:53:23Z)
• SafeSwitch: Steering Unsafe LLM Behavior via Internal Activation Signals [51.49737867797442]
  Large language models (LLMs) exhibit exceptional capabilities across various tasks but also pose risks by generating harmful content.<n>We show that LLMs can similarly perform internal assessments about safety in their internal states.<n>We propose SafeSwitch, a framework that regulates unsafe outputs by utilizing the prober-based internal state monitor.
  arXiv  Detail & Related papers  (2025-02-03T04:23:33Z)
• Chained Tuning Leads to Biased Forgetting [20.181135590652985]
  We show that models trained on downstream tasks forget their safety tuning to a greater extent than models trained in the opposite order.<n>We show that forgetting disproportionately impacts safety information about certain groups.
  arXiv  Detail & Related papers  (2024-12-21T03:51:58Z)
• Look Before You Leap: Enhancing Attention and Vigilance Regarding Harmful Content with GuidelineLLM [53.79753074854936]
  Large language models (LLMs) are increasingly vulnerable to emerging jailbreak attacks.<n>This vulnerability poses significant risks to real-world applications.<n>We propose a novel defensive paradigm called GuidelineLLM.
  arXiv  Detail & Related papers  (2024-12-10T12:42:33Z)
• Robust LLM safeguarding via refusal feature adversarial training [15.76605079209956]
  Large language models (LLMs) are vulnerable to adversarial attacks that can elicit harmful responses.<n>We propose Refusal Feature Adrial Training (ReFAT), a novel algorithm that efficiently performs adversarial training.<n>Experiment results show that ReFAT significantly improves the robustness of three popular LLMs against a wide range of adversarial attacks.
  arXiv  Detail & Related papers  (2024-09-30T08:41:39Z)
• SLM as Guardian: Pioneering AI Safety with Small Language Models [6.799423428734095]
  Internalizing safeguard features into larger models brought challenges of higher training cost and unintended degradation of helpfulness. In this paper, we leverage a smaller LLM for both harmful query detection and safeguard response generation. We demonstrate the effectiveness of our approach, providing on par or surpassing harmful query detection and safeguard response performance compared to the publicly available LLMs.
  arXiv  Detail & Related papers  (2024-05-30T08:03:15Z)
• ROSE Doesn't Do That: Boosting the Safety of Instruction-Tuned Large Language Models with Reverse Prompt Contrastive Decoding [89.0074567748505]
  We present reverse prompt contrastive decoding (ROSE), a simple-yet-effective method to boost the safety of existing instruction-tuned LLMs without any additional training. Experiments on 6 safety and 2 general-purpose tasks show that, our ROSE not only brings consistent and significant safety improvements (up to +13.8% safety score) upon 5 types of instruction-tuned LLMs, but also benefits the general-purpose ability of LLMs.
  arXiv  Detail & Related papers  (2024-02-19T06:58:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.