Deactivating Refusal Triggers: Understanding and Mitigating Overrefusal in Safety Alignment

• URL: http://arxiv.org/abs/2603.11388v1
• Date: Thu, 12 Mar 2026 00:07:19 GMT
• Title: Deactivating Refusal Triggers: Understanding and Mitigating Overrefusal in Safety Alignment
• Abstract summary: Safety alignment aims to ensure that large language models (LLMs) refuse harmful requests by post-training on harmful queries paired with refusal answers.<n>In this paper, we examine how overrefusal arises under safety alignment, and propose a mitigation strategy inspired by our findings.
• Score: 14.672527265339722
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Safety alignment aims to ensure that large language models (LLMs) refuse harmful requests by post-training on harmful queries paired with refusal answers. Although safety alignment is widely adopted in industry, the overrefusal problem where aligned LLMs also reject benign queries after safety alignment post-training, remains insufficiently studied. Such an issue degrades the usability of safety alignment in real-world applications. In this paper, we examine how overrefusal arises under safety alignment, and propose a mitigation strategy inspired by our findings. We define refusal triggers as linguistic cues in the training data that elicit refusal responses, safety alignment encourages LLMs to associate refusal triggers within a training sample with refusal responses, leading aligned LLMs to refuse harmful queries. However, the refusal triggers include not only harmful linguistic cues but also non-harmful cues, therefore causing overrefusal to benign queries. Building on this mechanistic analysis, we propose a method that explicitly considers refusal triggers in the safety alignment fine-tuning. Empirical results demonstrate that our approach achieves a more favorable trade-off between defense against jailbreak attacks and responsiveness to benign queries, outperforming prior methods. Warning: this paper contains harmful and biased sentences.

Related papers

• Refuse without Refusal: A Structural Analysis of Safety-Tuning Responses for Reducing False Refusals in Language Models [7.073710587892435]
  We show that refusal statements impede accurate discrimination between harmful and benign queries by inducing reliance on superficial cues.<n>We also show that training solely on rationales reduces false refusals while maintaining a comparable level of safety performance.
  arXiv  Detail & Related papers  (2026-09-04T04:37:10Z)
• Safety in Batches? Understanding and Mitigating Safety Failures in Batch Prompting [31.723003820235448]
  We show that a harmful question reliably refused in isolation can elicit a harmful response when embedded in a batch of benign questions.<n>We identify this as a distinct safety failure mode, not reducible to known vulnerabilities such as in-context learning or long-context effects.<n>We show that batch-aware preference optimization effectively mitigates the vulnerability.
  arXiv  Detail & Related papers  (2026-08-03T01:07:56Z)
• Beyond "I cannot fulfill this request": Alleviating Rigid Rejection in LLMs via Label Enhancement [39.705063586885075]
  Large Language Models (LLMs) rely on safety alignment to obey safe requests while refusing harmful ones.<n>This paper proposes LANCE to ensure safe yet flexible and natural responses via label enhancement.<n> Experiments demonstrate that LANCE significantly alleviates the rigid rejection problem while maintaining high security standards.
  arXiv  Detail & Related papers  (2026-05-08T15:33:54Z)
• Learning to Extract Context for Context-Aware LLM Inference [60.376872353918394]
  User prompts to large language models (LLMs) are often ambiguous or under-specified.<n> contextual cues shaped by user intentions, prior knowledge, and risk factors influence what constitutes an appropriate response.<n>We propose a framework that extracts and leverages such contextual information from the user prompt itself.
  arXiv  Detail & Related papers  (2025-12-12T19:10:08Z)
• When Models Outthink Their Safety: Mitigating Self-Jailbreak in Large Reasoning Models with Chain-of-Guardrails [74.63933201261595]
  Large Reasoning Models (LRMs) demonstrate remarkable capabilities on complex reasoning tasks.<n>LRMs remain vulnerable to severe safety risks, including harmful content generation and jailbreak attacks.<n>We propose the Chain-of-Guardrail (CoG), a training framework that recomposes or backtracks unsafe reasoning steps.
  arXiv  Detail & Related papers  (2025-10-24T09:32:25Z)
• Beyond Over-Refusal: Scenario-Based Diagnostics and Post-Hoc Mitigation for Exaggerated Refusals in LLMs [10.896368527058714]
  Large language models (LLMs) frequently produce false refusals, declining benign requests that contain terms resembling unsafe queries.<n>We introduce two comprehensive benchmarks: the Exaggerated Safety Benchmark (XSB) for single-turn prompts, annotated with "Focus" keywords that identify refusal-inducing triggers, and the Multi-turn Scenario-based Exaggerated Safety Benchmark (MS-XSB)<n>Our benchmarks reveal that exaggerated refusals persist across diverse recent LLMs and are especially pronounced in complex, multi-turn scenarios.
  arXiv  Detail & Related papers  (2025-10-09T12:38:16Z)
• Toward Safer Diffusion Language Models: Discovery and Mitigation of Priming Vulnerability [5.650647159993238]
  Diffusion language models (DLMs) generate tokens in parallel through iterative denoising.<n>In this paper, we reveal that DLMs have a critical vulnerability stemming from their iterative denoising process.<n>We propose a novel safety alignment method tailored to DLMs that trains models to generate safe responses from contaminated intermediate states.
  arXiv  Detail & Related papers  (2025-10-01T06:35:23Z)
• Revisiting Backdoor Attacks on LLMs: A Stealthy and Practical Poisoning Framework via Harmless Inputs [54.90315421117162]
  We propose a novel poisoning method via completely harmless data.<n>Inspired by the causal reasoning in auto-regressive LLMs, we aim to establish robust associations between triggers and an affirmative response prefix.<n>We observe an interesting resistance phenomenon where the LLM initially appears to agree but subsequently refuses to answer.
  arXiv  Detail & Related papers  (2025-05-23T08:13:59Z)
• Think Before Refusal : Triggering Safety Reflection in LLMs to Mitigate False Refusal Behavior [59.20260988638777]
  We demonstrate that prompting safety reflection before generating a response can mitigate false refusal behavior.<n>In an ablation study across 15 pre-trained models, we show that models fine-tuned with safety reflection significantly reduce false refusal behavior.
  arXiv  Detail & Related papers  (2025-03-22T23:35:49Z)
• Safety is Not Only About Refusal: Reasoning-Enhanced Fine-tuning for Interpretable LLM Safety [41.32331563680919]
  Large Language Models (LLMs) are vulnerable to jailbreak attacks that exploit weaknesses in traditional safety alignment.<n>We propose Reasoning-enhanced Finetuning for interpretable LLM Safety (Rational)<n>Rational trains models to engage in explicit safe reasoning before response.
  arXiv  Detail & Related papers  (2025-03-06T22:47:45Z)
• SCANS: Mitigating the Exaggerated Safety for LLMs via Safety-Conscious Activation Steering [56.92068213969036]
  Safety alignment is indispensable for Large Language Models (LLMs) to defend threats from malicious instructions.<n>Recent researches reveal safety-aligned LLMs prone to reject benign queries due to the exaggerated safety issue.<n>We propose a Safety-Conscious Activation Steering (SCANS) method to mitigate the exaggerated safety concerns.
  arXiv  Detail & Related papers  (2024-08-21T10:01:34Z)
• Refuse Whenever You Feel Unsafe: Improving Safety in LLMs via Decoupled Refusal Training [67.30423823744506]
  We introduce a novel approach, Decoupled Refusal Training (DeRTa), designed to empower LLMs to refuse compliance to harmful prompts at any response position.<n>DeRTa incorporates two novel components: (1) Maximum Likelihood Estimation with Harmful Response Prefix, which trains models to recognize and avoid unsafe content by appending a segment of harmful response to the beginning of a safe response, and (2) Reinforced Transition Optimization (RTO), which equips models with the ability to transition from potential harm to safety refusal consistently throughout the harmful response sequence.
  arXiv  Detail & Related papers  (2024-07-12T09:36:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.