What Drives Representation Steering? A Mechanistic Case Study on Steering Refusal

• URL: http://arxiv.org/abs/2604.08524v1
• Date: Thu, 09 Apr 2026 17:57:14 GMT
• Title: What Drives Representation Steering? A Mechanistic Case Study on Steering Refusal
• Abstract summary: We show that different steering methodologies leverage functionally interchangeable circuits when applied at the same layer.<n>We show that steering vectors can be sparsified by up to 90-99% while retaining most performance.
• Score: 53.189667624047416
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Applying steering vectors to large language models (LLMs) is an efficient and effective model alignment technique, but we lack an interpretable explanation for how it works-- specifically, what internal mechanisms steering vectors affect and how this results in different model outputs. To investigate the causal mechanisms underlying the effectiveness of steering vectors, we conduct a comprehensive case study on refusal. We propose a multi-token activation patching framework and discover that different steering methodologies leverage functionally interchangeable circuits when applied at the same layer. These circuits reveal that steering vectors primarily interact with the attention mechanism through the OV circuit while largely ignoring the QK circuit-- freezing all attention scores during steering drops performance by only 8.75% across two model families. A mathematical decomposition of the steered OV circuit further reveals semantically interpretable concepts, even in cases where the steering vector itself does not. Leveraging the activation patching results, we show that steering vectors can be sparsified by up to 90-99% while retaining most performance, and that different steering methodologies agree on a subset of important dimensions.

Related papers

• Disentangling Steering Vectors [13.671243729016568]
  We propose a framework to explicitly isolate individual and semantically consistent features from composite directions.<n>Specifically, we pair positive and negative activations and take their differences to generate a set of instance-level steering vectors.<n>We show that our method yields a set of semantically consistent basis vectors whose steering effects are mutually distinguishable.
  arXiv  Detail & Related papers  (2026-09-07T04:45:32Z)
• When is Your LLM Steerable? [56.656180566692946]
  Activation steering offers a lightweight approach to control language models' behavior at inference time.<n>Finding the regime and boundaries of successful steering typically requires expensive grid searches.<n>We investigate whether steerability can be predicted from the model's internal states at the beginning of the generation process.
  arXiv  Detail & Related papers  (2026-06-10T02:55:34Z)
• Activation Steering Induces Emergent Misalignment: A More Comprehensive Evaluation [74.17379276939599]
  We show that activation steering can induce broad misalignment, even in the recent Qwen-3.5 series.<n>We characterize properties of AS-induced EM by analyzing key steering-specific factors, including steering magnitude, the low-rank structure of the steering subspace, and the number of epochs during steering-vector construction.
  arXiv  Detail & Related papers  (2026-06-07T15:34:59Z)
• Understanding Unreliability of Steering Vectors in Language Models: Geometric Predictors and the Limits of Linear Approximations [0.0]
  I investigate why steering reliability differs across behaviors and how it is impacted by steering vector training data.<n>I find that higher cosine similarity between training activation differences predicts more reliable steering.<n>I observe that behavior datasets where positive and negative activations are better separated along the steering direction are more reliably steerable.
  arXiv  Detail & Related papers  (2026-02-19T22:37:05Z)
• Steering Latent Traits, Not Learned Facts: An Empirical Study of Activation Control Limits [0.0]
  Large language models (LLMs) require precise behavior control for safe and effective deployment across diverse applications.<n>We focus on the question of how steering effectiveness varies across different behavior types and whether the nature of target behaviors can predict steering success.
  arXiv  Detail & Related papers  (2025-11-23T04:28:41Z)
• KV Cache Steering for Controlling Frozen LLMs [80.50365534625438]
  cache steering is a lightweight method for implicit steering of language models.<n>We apply cache steering to induce chain-of-thought reasoning in small language models.
  arXiv  Detail & Related papers  (2025-07-11T17:59:36Z)
• Understanding (Un)Reliability of Steering Vectors in Language Models [21.33093425619501]
  This paper studies the influence of prompt types and the geometry of activation differences on steering reliability.<n>We find that all seven prompt types used in our experiments produce a net positive steering effect, but exhibit high variance across samples, and often give an effect opposite of the desired one.
  arXiv  Detail & Related papers  (2025-05-28T17:53:31Z)
• Beyond Prompt Engineering: Robust Behavior Control in LLMs via Steering Target Atoms [71.85633762642125]
  The vast number of parameters in models often results in highly intertwined internal representations.<n>Recent research has explored the use of sparse autoencoders (SAE) to disentangle knowledge in high-dimensional spaces for steering.<n>We propose Steering Target Atoms (STA), a novel method that isolates and manipulates disentangled knowledge components to enhance safety.
  arXiv  Detail & Related papers  (2025-05-23T17:59:18Z)
• Activation Scaling for Steering and Interpreting Language Models [55.59689963561315]
  We argue that successfully intervening on a model is a prerequisite for interpreting its internal workings. We establish a three-term objective: a successful intervention should flip the correct with the wrong token and vice versa. Using gradient-based optimization, this objective lets us learn (and later evaluate) a specific kind of efficient and interpretable intervention.
  arXiv  Detail & Related papers  (2024-10-07T12:01:32Z)
• Analyzing the Generalization and Reliability of Steering Vectors [8.253773195379166]
  We show that steering vectors have substantial limitations both in- and out-of-distribution.<n>In-distribution, steerability is highly variable across different inputs.<n>Out-of-distribution, while steering vectors often generalise well, for several concepts they are brittle to reasonable changes in the prompt.
  arXiv  Detail & Related papers  (2024-07-17T08:32:03Z)
• Control-Aware Prediction Objectives for Autonomous Driving [78.19515972466063]
  We present control-aware prediction objectives (CAPOs) to evaluate the downstream effect of predictions on control without requiring the planner be differentiable. We propose two types of importance weights that weight the predictive likelihood: one using an attention model between agents, and another based on control variation when exchanging predicted trajectories for ground truth trajectories.
  arXiv  Detail & Related papers  (2022-04-28T07:37:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.