Beyond Input Activations: Identifying Influential Latents by Gradient Sparse Autoencoders

• URL: http://arxiv.org/abs/2505.08080v1
• Date: Mon, 12 May 2025 21:29:12 GMT
• Title: Beyond Input Activations: Identifying Influential Latents by Gradient Sparse Autoencoders
• Abstract summary: This work is built on two key hypotheses: (1) activated latents do not contribute equally to the construction of the model's output, and (2) only latents with high causal influence are effective for model steering.<n>To validate these hypotheses, we propose Gradient Sparse Autoencoder (GradSAE), a simple yet effective method that identifies the most influential latents by incorporating output-side gradient information.
• Score: 38.53091987125025
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Sparse Autoencoders (SAEs) have recently emerged as powerful tools for interpreting and steering the internal representations of large language models (LLMs). However, conventional approaches to analyzing SAEs typically rely solely on input-side activations, without considering the causal influence between each latent feature and the model's output. This work is built on two key hypotheses: (1) activated latents do not contribute equally to the construction of the model's output, and (2) only latents with high causal influence are effective for model steering. To validate these hypotheses, we propose Gradient Sparse Autoencoder (GradSAE), a simple yet effective method that identifies the most influential latents by incorporating output-side gradient information.

Related papers

• Applied Explainability for Large Language Models: A Comparative Study [0.0]
  This paper presents an applied comparative study of three explainability techniques: Integrated Gradients, Attention Rollout, and SHAP.<n>The results show that gradient-based attribution provides more stable and intuitive explanations.<n>Work highlights key trade-offs between explainability methods and emphasizes their role as diagnostic tools rather than definitive explanations.
  arXiv  Detail & Related papers  (2026-04-15T13:07:29Z)
• Dual Path Attribution: Efficient Attribution for SwiGLU-Transformers through Layer-Wise Target Propagation [5.186807923082922]
  We introduce Dual Path Attribution (DPA), a novel framework that faithfully traces information flow on the frozen transformer in one forward and one backward pass.<n>DPA achieves state-of-the-art faithfulness and unprecedented efficiency compared to existing baselines.
  arXiv  Detail & Related papers  (2026-03-20T08:28:24Z)
• Mechanistic Knobs in LLMs: Retrieving and Steering High-Order Semantic Features via Sparse Autoencoders [8.188989044347595]
  We propose a Sparse Autoencoder-based framework for retrieving and steering semantically interpretable internal features.<n>Using the Big Five personality traits as a case study, we demonstrate that our method enables precise, bidirectional steering of model behavior.
  arXiv  Detail & Related papers  (2026-01-06T12:40:37Z)
• A Comparative Analysis of Sparse Autoencoder and Activation Difference in Language Model Steering [0.0]
  We propose focusing on a single, most relevant SAE latent (top-1), eliminating redundant features.<n>We show that steering an SAE latent associated with reasoning reliably elicits step-by-step mathematical reasoning.<n>Our results demonstrate that SAEs outperform mean activation difference methods on mathematical reasoning benchmarks and match their performance on IF-Eval.
  arXiv  Detail & Related papers  (2025-09-24T08:31:31Z)
• GrAInS: Gradient-based Attribution for Inference-Time Steering of LLMs and VLMs [56.93583799109029]
  GrAInS is an inference-time steering approach that operates across both language-only and vision-language models and tasks.<n>During inference, GrAInS hidden activations at transformer layers guided by token-level attribution signals, and normalizes activations to preserve representational scale.<n>It consistently outperforms both fine-tuning and existing steering baselines.
  arXiv  Detail & Related papers  (2025-07-24T02:34:13Z)
• Exploring and Exploiting the Inherent Efficiency within Large Reasoning Models for Self-Guided Efficiency Enhancement [101.77467538102924]
  Large reasoning models (LRMs) exhibit overthinking, which hinders efficiency and inflates inference cost.<n>We propose two lightweight methods to enhance LRM efficiency.<n>First, we introduce Efficiency Steering, a training-free activation steering technique that modulates reasoning behavior via a single direction.<n>Second, we develop Self-Rewarded Efficiency RL, a reinforcement learning framework that dynamically balances task accuracy and brevity.
  arXiv  Detail & Related papers  (2025-06-18T17:18:12Z)
• SAEs Are Good for Steering -- If You Select the Right Features [45.47261543304217]
  Current methods identify SAE features to steer by analyzing the input tokens that activate them.<n>In this work, we draw a distinction between two types of features: input features, which mainly capture patterns in the model's input, and output features, which have a human-understandable effect on the model's output.
  arXiv  Detail & Related papers  (2025-05-26T14:47:59Z)
• Patterns and Mechanisms of Contrastive Activation Engineering [0.374490703387131]
  CAE has the potential to introduce a new paradigm of flexible, task-specific behavior tuning.<n>We analyze the performance of CAE in in in-distribution, out-of-distribution settings, evaluate drawbacks, and begin to develop comprehensive guidelines for its effective deployment.
  arXiv  Detail & Related papers  (2025-05-06T05:15:12Z)
• LF-Steering: Latent Feature Activation Steering for Enhancing Semantic Consistency in Large Language Models [16.37602070339033]
  Large Language Models (LLMs) often generate inconsistent responses when prompted with semantically equivalent paraphrased inputs.<n>We propose LF-Steering, a novel activation steering approach to precisely identify latent feature representations responsible for semantic inconsistency.<n>Our method maps the hidden states of the relevant transformer layer into a sparsely activated, high-dimensional feature space based on a sparse autoencoder.
  arXiv  Detail & Related papers  (2025-01-19T13:06:51Z)
• Features that Make a Difference: Leveraging Gradients for Improved Dictionary Learning [4.051777802443125]
  Sparse Autoencoders (SAEs) are a promising approach for extracting neural network representations.<n>We introduce Gradient SAEs, which modify the $k$-sparse autoencoder architecture by augmenting the TopK activation function.<n>We find evidence that g-SAEs learn latents that are on average more effective at steering models in arbitrary contexts.
  arXiv  Detail & Related papers  (2024-11-15T18:03:52Z)
• Ladder-of-Thought: Using Knowledge as Steps to Elevate Stance Detection [73.31406286956535]
  We introduce the Ladder-of-Thought (LoT) for the stance detection task. LoT directs the small LMs to assimilate high-quality external knowledge, refining the intermediate rationales produced. Our empirical evaluations underscore LoT's efficacy, marking a 16% improvement over GPT-3.5 and a 10% enhancement compared to GPT-3.5 with CoT on stance detection task.
  arXiv  Detail & Related papers  (2023-08-31T14:31:48Z)
• Disentanglement via Latent Quantization [60.37109712033694]
  In this work, we construct an inductive bias towards encoding to and decoding from an organized latent space. We demonstrate the broad applicability of this approach by adding it to both basic data-re (vanilla autoencoder) and latent-reconstructing (InfoGAN) generative models.
  arXiv  Detail & Related papers  (2023-05-28T06:30:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.