The Rogue Scalpel: Activation Steering Compromises LLM Safety

• URL: http://arxiv.org/abs/2509.22067v1
• Date: Fri, 26 Sep 2025 08:49:47 GMT
• Title: The Rogue Scalpel: Activation Steering Compromises LLM Safety
• Abstract summary: Activation steering is a technique for controlling LLM behavior by adding semantically meaningful vectors directly into a model's hidden states during inference.<n>We demonstrate the opposite: steering systematically breaks model alignment safeguards, making it comply with harmful requests.
• Score: 11.402179030703188
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Activation steering is a promising technique for controlling LLM behavior by adding semantically meaningful vectors directly into a model's hidden states during inference. It is often framed as a precise, interpretable, and potentially safer alternative to fine-tuning. We demonstrate the opposite: steering systematically breaks model alignment safeguards, making it comply with harmful requests. Through extensive experiments on different model families, we show that even steering in a random direction can increase the probability of harmful compliance from 0% to 2-27%. Alarmingly, steering benign features from a sparse autoencoder (SAE), a common source of interpretable directions, increases these rates by a further 2-4%. Finally, we show that combining 20 randomly sampled vectors that jailbreak a single prompt creates a universal attack, significantly increasing harmful compliance on unseen requests. These results challenge the paradigm of safety through interpretability, showing that precise control over model internals does not guarantee precise control over model behavior.

Related papers

• Safety Cost of Steering Vectors Is Separable and Reducible [15.133262969401004]
  Steering vectors can unintentionally compromise a model's safety mechanisms and increase compliance with harmful requests.<n>We show that this safety degradation arises from a separable component in the vector that disrupts the model's safety mechanisms but contributes little to the steering objective.<n>Our method substantially reduces steering-induced safety degradation while preserving the original steering effect with minimal impact on false refusal.
  arXiv  Detail & Related papers  (2026-08-09T00:29:54Z)
• Asymmetric Collapse in Model Merging: When Refusal Over- writes Recognition [4.800831341795478]
  We use two Gemma-3-1B-IT finetunes on two complementary safety objectives: CARES harm-level classification and WildJailbreak adversarial refusal.<n>We merge the two fine-tunes using Linear, SLERP, TIES, and DARE-TIES, and evaluate the merged models on classification accuracy, attack resistance, and benign compliance.
  arXiv  Detail & Related papers  (2026-07-26T19:10:34Z)
• The Geometry of Refusal: Linear Instability in Safety-Aligned LLMs [1.4323566945483497]
  We introduce Contrastive Logit Steering ( CLS), a zero-optimization framework that isolates the "refusal direction"<n> CLS operates directly on the output distribution, serving as a diagnostic probe for alignment.<n>Our experiments on 7 model families reveal that safety implementation is architecturally deterministic.
  arXiv  Detail & Related papers  (2026-06-21T22:04:48Z)
• CIAware-Bench: Benchmarking Control Intervention Awareness Across Frontier LLMs [100.38986535324284]
  We introduce textbfCIAware-Bench, a benchmark for measuring textbfcontrol textbfintervention (CI) awareness across frontier models.<n>CIAware-Bench tests whether models can distinguish their own trajectories from those modified by a control intervention.
  arXiv  Detail & Related papers  (2026-06-09T16:24:16Z)
• Analysing the Safety Pitfalls of Steering Vectors [19.424564915366847]
  Activation steering has emerged as a powerful tool to shape LLM behavior without the need for weight updates.<n>In this work, we present a systematic safety audit of steering vectors obtained with Contrastive Activation Addition (CAA)<n>We show that steering vectors consistently influence the success rate of jailbreak attacks, with stronger amplification under simple template-based attacks.
  arXiv  Detail & Related papers  (2026-03-25T17:16:11Z)
• Principled Steering via Null-space Projection for Jailbreak Defense in Vision-Language Models [71.11171576590134]
  Vision-language models can be easily induced by visual jailbreak attacks to generate harmful content.<n>We propose NullSteer, a null-space projected activation defense framework.<n>We show that NullSteer significantly reduces harmful outputs under various jailbreak attacks.
  arXiv  Detail & Related papers  (2026-03-23T15:23:23Z)
• Steering Externalities: Benign Activation Steering Unintentionally Increases Jailbreak Risk for Large Language Models [62.16655896700062]
  Activation steering is a technique to enhance the utility of Large Language Models (LLMs)<n>We show that it unintentionally introduces critical and under-explored safety risks.<n>Experiments reveal that these interventions act as a force multiplier, creating new vulnerabilities to jailbreaks and increasing attack success rates to over 80% on standard benchmarks.
  arXiv  Detail & Related papers  (2026-02-03T12:32:35Z)
• Self-Guard: Defending Large Reasoning Models via enhanced self-reflection [54.775612141528164]
  Self-Guard is a lightweight safety defense framework for Large Reasoning Models.<n>It bridges the awareness-compliance gap, achieving robust safety performance without compromising model utility.<n>Self-Guard exhibits strong generalization across diverse unseen risks and varying model scales.
  arXiv  Detail & Related papers  (2026-01-31T13:06:11Z)
• DiffuGuard: How Intrinsic Safety is Lost and Found in Diffusion Large Language Models [50.21378052667732]
  We conduct an in-depth analysis of dLLM vulnerabilities to jailbreak attacks across two distinct dimensions: intra-step and inter-step dynamics.<n>We propose DiffuGuard, a training-free defense framework that addresses vulnerabilities through a dual-stage approach.
  arXiv  Detail & Related papers  (2025-09-29T05:17:10Z)
• LatentGuard: Controllable Latent Steering for Robust Refusal of Attacks and Reliable Response Generation [4.29885665563186]
  LATENTGUARD is a framework that combines behavioral alignment with supervised latent space control for interpretable and precise safety steering.<n>Our results show significant improvements in both safety controllability and response interpretability without compromising utility.
  arXiv  Detail & Related papers  (2025-09-24T07:31:54Z)
• Turning the Spell Around: Lightweight Alignment Amplification via Rank-One Safety Injection [47.347413305965006]
  Safety alignment in Large Language Models (LLMs) often involves mediating internal representations to refuse harmful requests.<n>Recent research has demonstrated that these safety mechanisms can be bypassed by ablating or removing specific representational directions.<n>We propose Rank-One Safety Injection (ROSI), a white-box method that amplifies a model's safety alignment by permanently steering its activations toward the refusal-mediating subspace.
  arXiv  Detail & Related papers  (2025-08-28T13:22:33Z)
• Watch the Weights: Unsupervised monitoring and control of fine-tuned LLMs [14.779177849006963]
  We introduce a new method for understanding, monitoring and controlling fine-tuned large language models (LLMs)<n>We demonstrate that the top singular of the weight difference between a fine-tuned model and its base model correspond to newly acquired behaviors.<n>For backdoored models that bypasses safety mechanisms when a secret trigger is present, our method stops up to 100% of attacks with a false positive rate below 1.2%.
  arXiv  Detail & Related papers  (2025-07-31T21:04:12Z)
• Anomalous Decision Discovery using Inverse Reinforcement Learning [3.3675535571071746]
  Anomaly detection plays a critical role in Autonomous Vehicles (AVs) by identifying unusual behaviors through perception systems.<n>Current approaches, which often rely on predefined thresholds or supervised learning paradigms, exhibit reduced efficacy when confronted with unseen scenarios.<n>We present Trajectory-Reward Guided Adaptive Pre-training (TRAP), a novel IRL framework for anomaly detection.
  arXiv  Detail & Related papers  (2025-07-06T17:01:02Z)
• AdaSteer: Your Aligned LLM is Inherently an Adaptive Jailbreak Defender [99.3105257001476]
  We propose AdaSteer, an adaptive activation steering method that adjusts model behavior based on input characteristics.<n>AdaSteer steers input representations along both the Rejection Direction (RD) and Harmfulness Direction (HD)<n>Our results highlight the potential of interpretable model internals for real-time, flexible safety enforcement in LLMs.
  arXiv  Detail & Related papers  (2025-04-13T07:39:17Z)
• SCANS: Mitigating the Exaggerated Safety for LLMs via Safety-Conscious Activation Steering [56.92068213969036]
  Safety alignment is indispensable for Large Language Models (LLMs) to defend threats from malicious instructions.<n>Recent researches reveal safety-aligned LLMs prone to reject benign queries due to the exaggerated safety issue.<n>We propose a Safety-Conscious Activation Steering (SCANS) method to mitigate the exaggerated safety concerns.
  arXiv  Detail & Related papers  (2024-08-21T10:01:34Z)
• Steering Without Side Effects: Improving Post-Deployment Control of Language Models [61.99293520621248]
  Language models (LMs) have been shown to behave unexpectedly post-deployment. We present KL-then-steer (KTS), a technique that decreases the side effects of steering while retaining its benefits. Our best method prevents 44% of jailbreak attacks compared to the original Llama-2-chat-7B model.
  arXiv  Detail & Related papers  (2024-06-21T01:37:39Z)
• On Prompt-Driven Safeguarding for Large Language Models [172.13943777203377]
  We find that in the representation space, the input queries are typically moved by safety prompts in a "higher-refusal" direction. Inspired by these findings, we propose a method for safety prompt optimization, namely DRO. Treating a safety prompt as continuous, trainable embeddings, DRO learns to move the queries' representations along or opposite the refusal direction, depending on their harmfulness.
  arXiv  Detail & Related papers  (2024-01-31T17:28:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.