SAEs Are Good for Steering -- If You Select the Right Features

• URL: http://arxiv.org/abs/2505.20063v1
• Date: Mon, 26 May 2025 14:47:59 GMT
• Title: SAEs Are Good for Steering -- If You Select the Right Features
• Abstract summary: Current methods identify SAE features to steer by analyzing the input tokens that activate them.<n>In this work, we draw a distinction between two types of features: input features, which mainly capture patterns in the model's input, and output features, which have a human-understandable effect on the model's output.
• Score: 45.47261543304217
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Sparse Autoencoders (SAEs) have been proposed as an unsupervised approach to learn a decomposition of a model's latent space. This enables useful applications such as steering - influencing the output of a model towards a desired concept - without requiring labeled data. Current methods identify SAE features to steer by analyzing the input tokens that activate them. However, recent work has highlighted that activations alone do not fully describe the effect of a feature on the model's output. In this work, we draw a distinction between two types of features: input features, which mainly capture patterns in the model's input, and output features, which have a human-understandable effect on the model's output. We propose input and output scores to characterize and locate these types of features, and show that high values for both scores rarely co-occur in the same features. These findings have practical implications: after filtering out features with low output scores, we obtain 2-3x improvements when steering with SAEs, making them competitive with supervised methods.

Related papers

• Sparse Autoencoders Encode Both Concepts and Functions: The Downstream Geometry of Feature Effects [58.98080129016082]
  Feature-Effect Geometry Analysis (FEGA) is an unsupervised framework that removes the same active SAE feature across contexts.<n>We show that a feature can be interpretable and causally relevant without providing a stable direction for steering.
  arXiv  Detail & Related papers  (2026-07-27T16:45:08Z)
• XtrAIn: Training-Guided Occlusion for Feature Attribution [3.3609128464006015]
  Occlusion-based attribution methods can estimate feature importance by perturbing input features and measuring the resulting change in model output.<n>We introduce XtrAIn, a training-guided attribution method that transfers the occlusion operation from the input space to the parameter space.<n>XtrAIn produces cleaner and more interpretable attribution patterns than standard attribution baselines.
  arXiv  Detail & Related papers  (2026-06-09T13:52:05Z)
• ADAG: Automatically Describing Attribution Graphs [50.460651620833055]
  We introduce textbfADAG, an end-to-end pipeline for describing these attribution graphs which is fully automated.<n>We then introduce a novel clustering algorithm for grouping features, and an LLM explainer--simulator setup which generates and scores natural-language explanations of the functional role of these feature groups.<n>We run our system on known human-analysed circuit-tracing tasks and recover interpretable circuits, and further show ADAG can find steerable clusters which are responsible for a harmful advice jailbreak in Llama 3.1 8B Instruct.
  arXiv  Detail & Related papers  (2026-04-08T21:34:37Z)
• EST: Towards Efficient Scaling Laws in Click-Through Rate Prediction via Unified Modeling [13.693397814262681]
  Efficiently scaling industrial Click-Through Rate (CTR) prediction has recently attracted significant research attention.<n>We propose the Efficiently Scalable Transformer (EST), which achieves fully unified modeling by processing all raw inputs in a single sequence without lossy aggregation.<n>EST significantly outperforms production baselines, delivering a 3.27% RPM (Revenue Per Mile) increase and a 1.22% CTR lift.
  arXiv  Detail & Related papers  (2026-02-11T12:51:54Z)
• Control Reinforcement Learning: Interpretable Token-Level Steering of LLMs via Sparse Autoencoder Features [1.5874067490843806]
  Control Reinforcement Learning trains a policy to select SAE features for steering at each token, producing interpretable intervention logs.<n> Adaptive Feature Masking encourages diverse feature discovery while preserving singlefeature interpretability.<n>On Gemma 2 2B across MMLU, BBQ, GSM8K, HarmBench, and XSTest, CRL achieves improvements while providing per-token intervention logs.
  arXiv  Detail & Related papers  (2026-02-11T02:28:49Z)
• ssToken: Self-modulated and Semantic-aware Token Selection for LLM Fine-tuning [51.133569963553576]
  ssToken is a Self-modulated and Semantic-aware Token Selection approach.<n>We show that both self-modulated selection and semantic-aware selection alone outperform full-data fine-tuning.
  arXiv  Detail & Related papers  (2025-10-21T03:21:04Z)
• CorrSteer: Generation-Time LLM Steering via Correlated Sparse Autoencoder Features [1.5874067490843806]
  We propose CorrSteer, which selects features by correlating sample correctness with SAE activations from generated tokens at inference time.<n>Our work establishes correlation-based selection as an effective and scalable approach for automated SAE steering across language model applications.
  arXiv  Detail & Related papers  (2025-08-18T00:01:42Z)
• Stochastic Encodings for Active Feature Acquisition [100.47043816019888]
  Active Feature Acquisition is an instance-wise, sequential decision making problem.<n>The aim is to dynamically select which feature to measure based on current observations, independently for each test instance.<n>Common approaches either use Reinforcement Learning, which experiences training difficulties, or greedily maximize the conditional mutual information of the label and unobserved features, which makes myopic.<n>We introduce a latent variable model, trained in a supervised manner. Acquisitions are made by reasoning about the features across many possible unobserved realizations in a latent space.
  arXiv  Detail & Related papers  (2025-08-03T23:48:46Z)
• GrAInS: Gradient-based Attribution for Inference-Time Steering of LLMs and VLMs [56.93583799109029]
  GrAInS is an inference-time steering approach that operates across both language-only and vision-language models and tasks.<n>During inference, GrAInS hidden activations at transformer layers guided by token-level attribution signals, and normalizes activations to preserve representational scale.<n>It consistently outperforms both fine-tuning and existing steering baselines.
  arXiv  Detail & Related papers  (2025-07-24T02:34:13Z)
• Beyond Input Activations: Identifying Influential Latents by Gradient Sparse Autoencoders [38.53091987125025]
  This work is built on two key hypotheses: (1) activated latents do not contribute equally to the construction of the model's output, and (2) only latents with high causal influence are effective for model steering.<n>To validate these hypotheses, we propose Gradient Sparse Autoencoder (GradSAE), a simple yet effective method that identifies the most influential latents by incorporating output-side gradient information.
  arXiv  Detail & Related papers  (2025-05-12T21:29:12Z)
• Enhancing Automated Interpretability with Output-Centric Feature Descriptions [14.591455690954513]
  We propose efficient, output-centric methods for automatically generating feature descriptions.<n>Our descriptions better capture the causal effect of a feature on model outputs than input-centric descriptions.<n>We show that output-centric descriptions can be used to find inputs that activate features previously thought to be "dead"
  arXiv  Detail & Related papers  (2025-01-14T18:53:00Z)
• Sparse Prototype Network for Explainable Pedestrian Behavior Prediction [60.80524827122901]
  We present Sparse Prototype Network (SPN), an explainable method designed to simultaneously predict a pedestrian's future action, trajectory, and pose. Regularized by mono-semanticity and clustering constraints, the prototypes learn consistent and human-understandable features.
  arXiv  Detail & Related papers  (2024-10-16T03:33:40Z)
• Stanceformer: Target-Aware Transformer for Stance Detection [59.69858080492586]
  Stance Detection involves discerning the stance expressed in a text towards a specific subject or target. Prior works have relied on existing transformer models that lack the capability to prioritize targets effectively. We introduce Stanceformer, a target-aware transformer model that incorporates enhanced attention towards the targets during both training and inference.
  arXiv  Detail & Related papers  (2024-10-09T17:24:28Z)
• Rethinking interpretation: Input-agnostic saliency mapping of deep visual classifiers [28.28834523468462]
  Saliency methods provide post-hoc model interpretation by attributing input features to the model outputs. We show that input-specific saliency mapping is intrinsically susceptible to misleading feature attribution. We introduce a new perspective of input-agnostic saliency mapping that computationally estimates the high-level features attributed by the model to its outputs.
  arXiv  Detail & Related papers  (2023-03-31T06:58:45Z)
• Tokenization Consistency Matters for Generative Models on Extractive NLP Tasks [54.306234256074255]
  We identify the issue of tokenization inconsistency that is commonly neglected in training generative models. This issue damages the extractive nature of these tasks after the input and output are tokenized inconsistently. We show that, with consistent tokenization, the model performs better in both in-domain and out-of-domain datasets.
  arXiv  Detail & Related papers  (2022-12-19T23:33:21Z)
• AutoAssign: Differentiable Label Assignment for Dense Object Detection [94.24431503373884]
  Auto COCO is an anchor-free detector for object detection. It achieves appearance-aware through a fully differentiable weighting mechanism. Our best model achieves 52.1% AP, outperforming all existing one-stage detectors.
  arXiv  Detail & Related papers  (2020-07-07T14:32:21Z)
• Adversarial Infidelity Learning for Model Interpretation [43.37354056251584]
  We propose a Model-agnostic Effective Efficient Direct (MEED) IFS framework for model interpretation. Our framework mitigates concerns about sanity, shortcuts, model identifiability, and information transmission. Our AIL mechanism can help learn the desired conditional distribution between selected features and targets.
  arXiv  Detail & Related papers  (2020-06-09T16:27:17Z)
• S3VAE: Self-Supervised Sequential VAE for Representation Disentanglement and Data Generation [31.38329747789168]
  We propose a sequential variational autoencoder to learn disentangled representations of sequential data under self-supervision. We exploit the benefits of some readily accessible supervisory signals from input data itself or some off-the-shelf functional models. Our model can easily disentangle the representation of an input sequence into static factors and dynamic factors.
  arXiv  Detail & Related papers  (2020-05-23T00:44:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.