Refuse without Refusal: A Structural Analysis of Safety-Tuning Responses for Reducing False Refusals in Language Models

• URL: http://arxiv.org/abs/2609.04714v1
• Date: Fri, 04 Sep 2026 04:37:10 GMT
• Title: Refuse without Refusal: A Structural Analysis of Safety-Tuning Responses for Reducing False Refusals in Language Models
• Abstract summary: We show that refusal statements impede accurate discrimination between harmful and benign queries by inducing reliance on superficial cues.<n>We also show that training solely on rationales reduces false refusals while maintaining a comparable level of safety performance.
• Score: 7.073710587892435
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Striking a balance between helpfulness and safety remains a fundamental challenge in aligning large language models. To achieve this balance, models should refuse harmful queries (e.g., "How do I shoot someone?") while remaining responsive to benign inputs, even those superficially resembling harmful queries (e.g., "Where can I shoot a good photo?"). However, models often struggle to distinguish genuinely harmful queries from benign queries that contain superficially risky language, resulting in false refusals. In this paper, we address the issue by decomposing a response in the safety-tuning dataset into two distinct components: (i) a boilerplate refusal statement and (ii) a rationale explaining the refusal. Our experiments and analyses show that refusal statements impede accurate discrimination between harmful and benign queries by inducing reliance on superficial cues. In contrast, training solely on rationales reduces false refusals while maintaining a comparable level of safety performance. Rationale-Only benefits also appear in our ICL configuration and remain compatible with the evaluated inference-time mitigation methods. The results emphasize the necessity of precisely curated, fine-grained safety supervision datasets and outline directions for constructing aligned agents that better reconcile helpfulness with safety.

Related papers

• Addressing Over-Refusal in LLMs with Competing Rewards [28.092622045521917]
  Safety training on language models often induces over-refusal.<n>We argue that unsafe reasoning can itself serve as a useful exploratory signal.<n>We show that this behavior helps over-refusal and defend against attacks that directly manipulate the reasoning to be harmful.
  arXiv  Detail & Related papers  (2026-06-30T14:38:49Z)
• Deactivating Refusal Triggers: Understanding and Mitigating Overrefusal in Safety Alignment [14.672527265339722]
  Safety alignment aims to ensure that large language models (LLMs) refuse harmful requests by post-training on harmful queries paired with refusal answers.<n>In this paper, we examine how overrefusal arises under safety alignment, and propose a mitigation strategy inspired by our findings.
  arXiv  Detail & Related papers  (2026-03-12T00:07:19Z)
• Understanding and Mitigating Over-refusal for Large Language Models via Safety Representation [23.989016366701232]
  Large language models harbor safety vulnerabilities.<n>Improvements in model safety often come at the cost of severe over-refusal.<n>We propose MOSR, designed to mitigate over-refusal by intervening the safety representation of LLMs.
  arXiv  Detail & Related papers  (2025-11-24T11:38:53Z)
• Refusal Falls off a Cliff: How Safety Alignment Fails in Reasoning? [68.82210578851442]
  We investigate why safety alignment fails in reasoning models through a mechanistic interpretability lens.<n>Using a linear probing approach to trace refusal intentions across token positions, we discover a phenomenon termed as textbfrefusal cliff<n>We propose textbfCliff-as-a-Judge, a novel data selection method that identifies training examples exhibiting the largest refusal cliff to efficiently repair reasoning models' safety alignment.
  arXiv  Detail & Related papers  (2025-10-07T15:32:59Z)
• A Granular Study of Safety Pretraining under Model Abliteration [64.24346997570275]
  We study model abliteration, a lightweight projection technique designed to remove refusal-sensitive directions.<n>We issue 100 prompts with balanced harmful and harmless cases, classify responses as **Refusal** or **Non-Refusal** using multiple judges, and validate judge fidelity.<n>Our study produces a checkpoint-level characterization of which data-centric safety components remain robust under abliteration.
  arXiv  Detail & Related papers  (2025-10-03T07:01:45Z)
• FalseReject: A Resource for Improving Contextual Safety and Mitigating Over-Refusals in LLMs via Structured Reasoning [12.467239356591238]
  FalseReject is a comprehensive resource containing 16k seemingly toxic queries accompanied by structured responses across 44 safety-related categories.<n>We propose a graph-informed adversarial multi-agent interaction framework to generate diverse and complex prompts.<n>We show that supervised finetuning with FalseReject substantially reduces unnecessary refusals without compromising overall safety or general language capabilities.
  arXiv  Detail & Related papers  (2025-05-12T20:45:25Z)
• Think Before Refusal : Triggering Safety Reflection in LLMs to Mitigate False Refusal Behavior [59.20260988638777]
  We demonstrate that prompting safety reflection before generating a response can mitigate false refusal behavior.<n>In an ablation study across 15 pre-trained models, we show that models fine-tuned with safety reflection significantly reduce false refusal behavior.
  arXiv  Detail & Related papers  (2025-03-22T23:35:49Z)
• Refuse Whenever You Feel Unsafe: Improving Safety in LLMs via Decoupled Refusal Training [67.30423823744506]
  We introduce a novel approach, Decoupled Refusal Training (DeRTa), designed to empower LLMs to refuse compliance to harmful prompts at any response position.<n>DeRTa incorporates two novel components: (1) Maximum Likelihood Estimation with Harmful Response Prefix, which trains models to recognize and avoid unsafe content by appending a segment of harmful response to the beginning of a safe response, and (2) Reinforced Transition Optimization (RTO), which equips models with the ability to transition from potential harm to safety refusal consistently throughout the harmful response sequence.
  arXiv  Detail & Related papers  (2024-07-12T09:36:33Z)
• On Prompt-Driven Safeguarding for Large Language Models [172.13943777203377]
  We find that in the representation space, the input queries are typically moved by safety prompts in a "higher-refusal" direction. Inspired by these findings, we propose a method for safety prompt optimization, namely DRO. Treating a safety prompt as continuous, trainable embeddings, DRO learns to move the queries' representations along or opposite the refusal direction, depending on their harmfulness.
  arXiv  Detail & Related papers  (2024-01-31T17:28:24Z)
• Navigating the OverKill in Large Language Models [84.62340510027042]
  We investigate the factors for overkill by exploring how models handle and determine the safety of queries. Our findings reveal the presence of shortcuts within models, leading to an over-attention of harmful words like 'kill' and prompts emphasizing safety will exacerbate overkill. We introduce Self-Contrastive Decoding (Self-CD), a training-free and model-agnostic strategy, to alleviate this phenomenon.
  arXiv  Detail & Related papers  (2024-01-31T07:26:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.