Signatures of Steerability in Activation Space of Language Models

• URL: http://arxiv.org/abs/2609.14151v1
• Date: Sat, 12 Sep 2026 21:06:55 GMT
• Title: Signatures of Steerability in Activation Space of Language Models
• Abstract summary: We show that simple separation metrics strongly correlate with the downstream steerability of language models across diverse settings.<n>Our results suggest that simple separability statistics can serve as practical diagnostics for when steering vectors are likely to work.
• Score: 3.7155332656584186
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Steering language models using a set of contrastive representations has been a canonical and computationally efficient method for controlling model behavior. Despite this success in controlling certain model behaviors, the effectiveness of activation steering varies markedly across concepts; the generalization properties of steering vectors are often considered a function of the dataset used to construct them. We make this dataset-dependence claim more rigorous and show that simple separation metrics strongly correlate with the downstream steerability of language models across diverse settings, even after controlling for layers and dataset effects. Beyond prediction, we provide evidence from a synthetic superposition experiment that separation metrics are strongly correlated with alignment between the empirical and true feature direction. Our results suggest that simple separability statistics can serve as practical diagnostics for when steering vectors are likely to work.

Related papers

• Detecting and Controlling Sycophancy with Cascading Linear Features [7.13923185919936]
  We present an iterative data generation pipeline that isolates cascading linear features responsible for a behavior.<n>We focus on detecting and steering away from sycophancy -- the tendency of language models to prioritize user validation.
  arXiv  Detail & Related papers  (2026-06-23T20:10:53Z)
• Understanding Unreliability of Steering Vectors in Language Models: Geometric Predictors and the Limits of Linear Approximations [0.0]
  I investigate why steering reliability differs across behaviors and how it is impacted by steering vector training data.<n>I find that higher cosine similarity between training activation differences predicts more reliable steering.<n>I observe that behavior datasets where positive and negative activations are better separated along the steering direction are more reliably steerable.
  arXiv  Detail & Related papers  (2026-02-19T22:37:05Z)
• Concept Influence: Leveraging Interpretability to Improve Performance and Efficiency in Training Data Attribution [11.387100835483672]
  Training Data Attribution (TDA) methods identify which training data drive specific behaviors, particularly unintended ones.<n>Existing approaches like influence functions are both computationally expensive and attribute based on single test examples.<n>We leverage interpretable structures within the model during the attribution.<n>We show that incorporating interpretable structure within traditional TDA pipelines can enable more scalable, explainable, and better control of model behavior through data.
  arXiv  Detail & Related papers  (2026-02-16T16:02:09Z)
• On the Identifiability of Steering Vectors in Large Language Models [0.0]
  Activation steering methods are widely used to control large language model behavior.<n>This interpretation implicitly assumes steering directions are identifiable and uniquely recoverable from input-output behavior.<n>We prove that steering vectors are fundamentally non-identifiable due to large equivalence classes of behaviorally indistinguishable interventions.
  arXiv  Detail & Related papers  (2026-02-06T15:53:50Z)
• Steering Language Models Before They Speak: Logit-Level Interventions [9.055997973281919]
  We propose a training-free inference-time logit intervention for controllable generation.<n>Our results show that statistically grounded logit steering can achieve large, consistent, and multi-task control gains.
  arXiv  Detail & Related papers  (2026-01-16T03:00:33Z)
• Nonparametric Data Attribution for Diffusion Models [57.820618036556084]
  Data attribution for generative models seeks to quantify the influence of individual training examples on model outputs.<n>We propose a nonparametric attribution method that operates entirely on data, measuring influence via patch-level similarity between generated and training images.
  arXiv  Detail & Related papers  (2025-10-16T03:37:16Z)
• KV Cache Steering for Controlling Frozen LLMs [80.50365534625438]
  cache steering is a lightweight method for implicit steering of language models.<n>We apply cache steering to induce chain-of-thought reasoning in small language models.
  arXiv  Detail & Related papers  (2025-07-11T17:59:36Z)
• One-shot Optimized Steering Vectors Mediate Safety-relevant Behaviors in LLMs [21.2431937128876]
  We propose optimizing steering vectors through gradient descent on a single training example.<n>We find that the resulting SVs effectively mediate safety-relevant behaviors in multiple models.<n>We extend work on "emergent misalignment" and show that SVs optimized to induce a model to write vulnerable code cause the model to respond harmfully.
  arXiv  Detail & Related papers  (2025-02-26T06:13:01Z)
• Activation Scaling for Steering and Interpreting Language Models [55.59689963561315]
  We argue that successfully intervening on a model is a prerequisite for interpreting its internal workings. We establish a three-term objective: a successful intervention should flip the correct with the wrong token and vice versa. Using gradient-based optimization, this objective lets us learn (and later evaluate) a specific kind of efficient and interpretable intervention.
  arXiv  Detail & Related papers  (2024-10-07T12:01:32Z)
• Revisiting Demonstration Selection Strategies in In-Context Learning [66.11652803887284]
  Large language models (LLMs) have shown an impressive ability to perform a wide range of tasks using in-context learning (ICL) In this work, we first revisit the factors contributing to this variance from both data and model aspects, and find that the choice of demonstration is both data- and model-dependent. We propose a data- and model-dependent demonstration selection method, textbfTopK + ConE, based on the assumption that textitthe performance of a demonstration positively correlates with its contribution to the model's understanding of the test samples.
  arXiv  Detail & Related papers  (2024-01-22T16:25:27Z)
• Disentanglement via Latent Quantization [60.37109712033694]
  In this work, we construct an inductive bias towards encoding to and decoding from an organized latent space. We demonstrate the broad applicability of this approach by adding it to both basic data-re (vanilla autoencoder) and latent-reconstructing (InfoGAN) generative models.
  arXiv  Detail & Related papers  (2023-05-28T06:30:29Z)
• Prototypical Representation Learning for Relation Extraction [56.501332067073065]
  This paper aims to learn predictive, interpretable, and robust relation representations from distantly-labeled data. We learn prototypes for each relation from contextual information to best explore the intrinsic semantics of relations. Results on several relation learning tasks show that our model significantly outperforms the previous state-of-the-art relational models.
  arXiv  Detail & Related papers  (2021-03-22T08:11:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.