On the Failure of Topic-Matched Contrast Baselines in Multi-Directional Refusal Abliteration

• URL: http://arxiv.org/abs/2603.22061v1
• Date: Mon, 23 Mar 2026 14:55:00 GMT
• Title: On the Failure of Topic-Matched Contrast Baselines in Multi-Directional Refusal Abliteration
• Abstract summary: This work investigates whether a topically matched contrast baseline yields superior refusal directions.<n>It was found that topic-matched contrast produces no functional refusal directions at any tested weight level.<n>The implications for the design of contrast baselines in abliteration research are discussed.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Inasmuch as the removal of refusal behavior from instruction-tuned language models by directional abliteration requires the extraction of refusal-mediating directions from the residual stream activation space, and inasmuch as the construction of the contrast baseline against which harmful prompt activations are compared has been treated in the existing literature as an implementation detail rather than a methodological concern, the present work investigates whether a topically matched contrast baseline yields superior refusal directions. The investigation is carried out on the Qwen~3.5 2B model using per-category matched prompt pairs, per-class Self-Organizing Map extraction, and Singular Value Decomposition orthogonalization. It was found that topic-matched contrast produces no functional refusal directions at any tested weight level on any tested layer, while unmatched contrast on the same model, same extraction code, and same evaluation protocol achieves complete refusal elimination on six layers. The geometric analysis of the failure establishes that topic-matched subtraction cancels the dominant activation component shared between harmful and harmless prompts of the same subject, reducing the extracted direction magnitude below the threshold at which weight-matrix projection perturbs the residual stream. The implications for the design of contrast baselines in abliteration research are discussed.

Related papers

• StateSwap: Probing Support-Elimination Hidden States in Multiple-Choice Questions [22.173138594307662]
  We investigate whether discrepancies arise from different internal representations induced by two framings.<n>We introduce a dual-framing protocol with minimally varied prompts.<n>We find that the two framings induce separable [STATE] activations concentrated in intermediate layers.
  arXiv  Detail & Related papers  (2026-09-01T11:13:26Z)
• Structuring Semantic Embeddings for Principle Evaluation: A Prototype-Guided Contrastive Learning Approach [58.15427952222424]
  General-purpose text embeddings are widely deployed for such tasks, but broad semantic similarity can place semantically similar yet task-distinct examples.<n>We introduce Prototype-Guided Contrastive Learning (PGCL), a prototype-guided geometric regularization module built on top of frozen text embeddings.
  arXiv  Detail & Related papers  (2026-08-15T13:19:25Z)
• Perturbation-based Regional Interpretability through Subtraction Mapping (PRISM): naming-error dissociations in language models and post-stroke aphasia [6.867794233852404]
  Subtraction analysis is the standard framework of human neuroimaging.<n>We develop PRISM (Perturbation-based Regional Interpretability through Subtraction Mapping)<n>PRISM gives a falsifiable, spatially resolved test of functional-specialization claims in transformer language models.
  arXiv  Detail & Related papers  (2026-08-13T02:02:05Z)
• R-APS: Compositional Reasoning and In-Context Meta-Learning for Constrained Design via Reflective Adversarial Pareto Search [18.8323743697237]
  Large language models (LLMs) are fluent on open-ended tasks, yet in agentic settings, fluency does not ensure reliable delivery.<n>We trace this gap to three coupled structural failures: errors propagate without localization, worst-case perturbations go unevaluated, and accumulated knowledge is never invalidated.<n>We argue these share a root cause: abductive, counterfactual, meta-inductive, corrective, and inductive reasoning pull a shared context in incompatible directions.
  arXiv  Detail & Related papers  (2026-06-03T12:45:39Z)
• When Hard Negatives Hurt: Bridging the Generative-Discriminative Gap in Hard Negative Synthesis for Retrieval [45.5843471557695]
  We show that naively incorporating generated negatives into contrastive learning often degrades retrieval performance.<n>Our analysis reveals two compounding failure modes: discriminative-agnostic generation and source-dependent shortcuts.<n>To close this gap, we propose CausalNeg consisting of two main modules.
  arXiv  Detail & Related papers  (2026-05-31T15:48:30Z)
• From Refusal Tokens to Refusal Control: Discovering and Steering Category-Specific Refusal Directions [7.599054770990825]
  We leverage a version of Llama 3 8B fine-tuned with categorical refusal tokens to enable inference-time control over fine-grained refusal behavior.<n>We construct categorical steering vectors with a lightweight probe that determines whether to steer toward or away from refusal during inference.<n>Across benchmarks, both categorical steering vectors and the low-rank combination consistently reduce over-refusals on benign prompts while increasing refusal rates on harmful prompts.
  arXiv  Detail & Related papers  (2026-03-09T06:37:16Z)
• On the Structural Non-Preservation of Epistemic Behaviour under Policy Transformation [51.56484100374058]
  We formalise such information-conditioned interaction patterns as behavioural dependency.<n>This induces a probe-relative notion of $$-behavioural equivalence and a within-policy behavioural distance.<n>Results identify structural conditions under which probe-conditioned behavioural separation is not preserved under common policy transformations.
  arXiv  Detail & Related papers  (2026-02-24T22:55:21Z)
• Adversarial Lens: Exploiting Attention Layers to Generate Adversarial Examples for Evaluation [0.2800045883776013]
  We generate adversarial examples directly from attention-layer token distributions.<n>We evaluate whether tokens extracted from intermediate layers can serve as effective adversarial perturbations for downstream evaluation tasks.
  arXiv  Detail & Related papers  (2025-12-29T19:59:52Z)
• Exploring Semantic-constrained Adversarial Example with Instruction Uncertainty Reduction [51.50282796099369]
  This paper develops a multi-dimensional instruction uncertainty reduction framework to generate semantically constrained adversarial examples.<n>By predicting the language-guided sampling process, the optimization process will be stabilized by the designed ResAdv-DDIM sampler.<n>We realize the reference-free generation of semantically constrained 3D adversarial examples for the first time.
  arXiv  Detail & Related papers  (2025-10-27T04:02:52Z)
• BURN: Backdoor Unlearning via Adversarial Boundary Analysis [73.14147934175604]
  Backdoor unlearning aims to remove backdoor-related information while preserving the model's original functionality.<n>We propose Backdoor Unlearning via adversaRial bouNdary analysis (BURN), a novel defense framework that integrates false correlation decoupling, progressive data refinement, and model purification.
  arXiv  Detail & Related papers  (2025-07-14T17:13:06Z)
• AMRFact: Enhancing Summarization Factuality Evaluation with AMR-Driven Negative Samples Generation [57.8363998797433]
  We propose AMRFact, a framework that generates perturbed summaries using Abstract Meaning Representations (AMRs) Our approach parses factually consistent summaries into AMR graphs and injects controlled factual inconsistencies to create negative examples, allowing for coherent factually inconsistent summaries to be generated with high error-type coverage.
  arXiv  Detail & Related papers  (2023-11-16T02:56:29Z)
• HiURE: Hierarchical Exemplar Contrastive Learning for Unsupervised Relation Extraction [63.64877263289319]
  Unsupervised relation extraction aims to extract the relationship between entities from natural language sentences without prior information on relational scope or distribution.<n>We propose a novel contrastive learning framework named HiURE, which has the capability to derive hierarchical signals from relational feature space using cross hierarchy attention.<n> Experimental results on two public datasets demonstrate the advanced effectiveness and robustness of HiURE on unsupervised relation extraction when compared with state-of-the-art models.
  arXiv  Detail & Related papers  (2022-05-04T17:56:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.