Related papers: Refusal Reads Only a Slice of What the Model Knows: Harm-Keyed Routing and Its Exceptions Across Model Families

Refusal Reads Only a Slice of What the Model Knows: Harm-Keyed Routing and Its Exceptions Across Model Families

URL: http://arxiv.org/abs/2609.14759v1
Date: Sun, 13 Sep 2026 19:49:44 GMT
Title: Refusal Reads Only a Slice of What the Model Knows: Harm-Keyed Routing and Its Exceptions Across Model Families
Abstract summary: We show that alignment applied after pretraining is shallow in a measurable way.<n>A single direction in a model's residual stream can be edited out, and the model stops refusing harmful requests.<n>We ask what it reads, and we separate that from what the model comprehends.
Score: 1.0829694003408499
License: http://creativecommons.org/licenses/by/4.0/
Abstract: Alignment applied after pretraining is shallow in a measurable way: a single direction in a model's residual stream can be edited out, and the model stops refusing harmful requests. That fact says how easily refusal can be removed, not what the refusal decision was reading in the first place. We ask what it reads, and we separate that from what the model comprehends. Across four open-weight models spanning three families, moral comprehension is native to pretraining: a low-rank moral subspace crystallizes during pretraining, and alignment rotates it once without rebuilding it. The refusal gate, in contrast, is a fresh post-training construction with only a weak pretraining precursor, written into a narrow control-token channel where the refusal decision is orthogonal to the moral-judgment decision. The central result is causal and comes from one model, OLMo-3. A nested interchange rank sweep patches successively larger slices of the moral subspace between matched requests and reads how much of refusal's response transfers: as the basis widens, moral judgment keeps reading more of it, while refusal levels off at the level of a single harm direction, and about three-quarters of refusal's causal input lies outside the moral subspace altogether. Refusal reads the harm percept, not the moral content that judgment reads on the same patches. The picture is not uniform across families. Llama reads broad moral content; Qwen reads beyond the single harm cue but is unresolved at our sample size; GPT-OSS reads harm, and its refusals can be argued in either direction by its own reasoning trace. Where refusal reads only a low-rank slice and routes around the bulk of what the model knows, a rank-one edit removes it. Whether widening what refusal reads would also deepen the behavior is the open question this raises.

Related papers

Refusal geometry reflects refusal training: diverse refusal prefixes can raise stable rank and weaken refusal vector ablation attacks [1.0152838128195467]
Refusal training protects AI models from jailbreaks by training models to decline unsafe queries, reducing the risk of misuse.<n>Recent work finds that refusal behavior in aligned language models can be mediated by a single activation direction or a low-dimensional refusal subspace shared across harmful prompts.<n>Yet it remains unclear why safety-critical features in a wide range of models emerge in a concentrated, low-dimensional structure.
arXiv Detail & Related papers (2026-08-26T05:35:54Z)
Abliteration Is Not a Scalpel: Off-Target Effects of Refusal Removal on Decision Disposition Across Model Families [51.56484100374058]
Abliteration - deleting a model's refusal direction from its weights - is the standard recipe behind popular "uncensored" open-weight models.<n>As a disposition probe we use 21,600 decisions under uncertainty, replayed through a frozen pipeline so the decision-layer model is the only variable.<n>Three effects replicate across both families (weeks-clustered CIs excluding zero)<n>A fourth effect reverses sign: the same operation makes Gemma-abliterated less confident and Qwen-abliterated more (family CIs non-overlapping)
arXiv Detail & Related papers (2026-07-19T22:27:00Z)
Breaking Refusal in the First Half: A Mechanistic Study of the Prefill Jailbreak [0.0]
Aligned language models refuse harmful requests, but a one-line prefill strips the refusal.<n>A linear probe reads harm as high as on the refused ones (0.91-0.98), while behavioral refusal drops to chance.<n>We localize it to an early window: a dose-matched position control shows the first half of the response suffices to break refusal, while the second half is nearly inert.
arXiv Detail & Related papers (2026-07-14T09:30:34Z)
Latent-space Attacks for Refusal Evasion in Language Models [14.290157825353846]
We recast refusal suppression as a latent-space evasion attack against linear probes trained to separate refused from answered prompts.<n>We achieve state-of-the-art attack success rate across 15 instruction-tuned, multimodal, and reasoning models.
arXiv Detail & Related papers (2026-05-20T20:10:27Z)
Refusal Falls off a Cliff: How Safety Alignment Fails in Reasoning? [68.82210578851442]
We investigate why safety alignment fails in reasoning models through a mechanistic interpretability lens.<n>Using a linear probing approach to trace refusal intentions across token positions, we discover a phenomenon termed as textbfrefusal cliff<n>We propose textbfCliff-as-a-Judge, a novel data selection method that identifies training examples exhibiting the largest refusal cliff to efficiently repair reasoning models' safety alignment.
arXiv Detail & Related papers (2025-10-07T15:32:59Z)
MORABLES: A Benchmark for Assessing Abstract Moral Reasoning in LLMs with Fables [50.29407048003165]
We present MORABLES, a human-verified benchmark built from fables and short stories drawn from historical literature.<n>The main task is structured as multiple-choice questions targeting moral inference, with carefully crafted distractors that challenge models to go beyond shallow, extractive question answering.<n>Our findings show that, while larger models outperform smaller ones, they remain susceptible to adversarial manipulation and often rely on superficial patterns rather than true moral reasoning.
arXiv Detail & Related papers (2025-09-15T19:06:10Z)
LLMs Encode Harmfulness and Refusal Separately [38.953626527935384]
LLMs are trained to refuse harmful instructions, but do they truly understand harmfulness beyond just refusing?<n>We identify a new dimension to analyze safety mechanisms in LLMs, i.e., harmfulness, which is encoded internally as a separate concept from refusal.<n>We find that certain jailbreak methods work by reducing the refusal signals without reversing the model's internal belief of harmfulness.
arXiv Detail & Related papers (2025-07-16T03:48:03Z)
An Embarrassingly Simple Defense Against LLM Abliteration Attacks [47.347413305965006]
A recent attack, termed abliteration, identifies and suppresses the single latent direction most responsible for refusal behavior.<n>We propose a defense that fundamentally alters how models express refusal.<n>Fine-tuning Llama-2-7B-Chat and Qwen2.5-Instruct yields models that maintain high refusal rates under abliteration.
arXiv Detail & Related papers (2025-05-25T09:18:24Z)
Think Before Refusal : Triggering Safety Reflection in LLMs to Mitigate False Refusal Behavior [59.20260988638777]
We demonstrate that prompting safety reflection before generating a response can mitigate false refusal behavior.<n>In an ablation study across 15 pre-trained models, we show that models fine-tuned with safety reflection significantly reduce false refusal behavior.
arXiv Detail & Related papers (2025-03-22T23:35:49Z)
Refusal Tokens: A Simple Way to Calibrate Refusals in Large Language Models [68.15108215197279]
A key component of building safe and reliable language models is enabling the models to appropriately refuse to answer certain questions.<n>We propose refusal tokens, one such token for each refusal category or a single refusal token, which are prepended to the model's responses during training.
arXiv Detail & Related papers (2024-12-09T18:40:44Z)
Refusal in Language Models Is Mediated by a Single Direction [4.532520427311685]
We show that refusal is mediated by a one-dimensional subspace, across 13 popular open-source chat models up to 72B parameters in size. We propose a novel white-box jailbreak method that surgically disables refusal with minimal effect on other capabilities.
arXiv Detail & Related papers (2024-06-17T16:36:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.