When Are Sparse Feature Interventions Actually Localized? Matched Evaluation for SAE-Based Safety Control

• URL: http://arxiv.org/abs/2607.10226v1
• Date: Sat, 11 Jul 2026 09:31:45 GMT
• Title: When Are Sparse Feature Interventions Actually Localized? Matched Evaluation for SAE-Based Safety Control
• Abstract summary: We evaluate when sparse autoencoder (SAE) features act as localized control handles for safety-relevant behavior.<n>We introduce a matched coherence-gated evaluation protocol for runtime safety interventions.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We evaluate when sparse autoencoder (SAE) features act as localized control handles for safety-relevant behavior. This question is difficult because apparent success can arise from weak interventions, mismatched baselines, model robustness, or degenerate outputs that automated safety judges mark as unsafe without representing meaningful harmful compliance. We introduce a matched coherence-gated evaluation protocol for runtime safety interventions: methods are compared at matched target-effect points, and the primary target metric counts harmful compliance only when an output is both judge-unsafe and coherent. Applying this protocol to three prompt splits on Gemma-2-9B-it with a Gemma Scope layer-20 residual SAE, we find that SAE feature ablation has a narrow useful regime. SAE top800 reaches a low-to-mid target effect with lower total perturbation and competitive utility, but SAE top1600 loses utility relative to a matched dense refusal-direction baseline, and SAE top3200 primarily induces coherence collapse. Human audit confirms that coherence gating removes unsafe-only artifacts, and feature diagnostics show that the useful regime is driven by a stable head of refusal-aligned features whose activation separation decays rapidly with rank. These results argue that SAE-based safety interventions should be evaluated as regime-dependent control mechanisms rather than assumed to be uniformly localized.

Related papers

• The Geometry of Refusal: Linear Instability in Safety-Aligned LLMs [1.4323566945483497]
  We introduce Contrastive Logit Steering ( CLS), a zero-optimization framework that isolates the "refusal direction"<n> CLS operates directly on the output distribution, serving as a diagnostic probe for alignment.<n>Our experiments on 7 model families reveal that safety implementation is architecturally deterministic.
  arXiv  Detail & Related papers  (2026-06-21T22:04:48Z)
• SAE Interventions are Unreliable: Post-Intervention Recovery of Suppressed Behavior [38.75847400495247]
  Sparse Autoencoders (SAEs) decompose residual-stream activations into interpretable features.<n>We show that clamping a specific harmful feature is expected to reliably prevent model misbehavior.<n>We formulate this vulnerability as post-intervention recovery, a constrained residual-space optimization problem.
  arXiv  Detail & Related papers  (2026-06-16T15:04:17Z)
• Action-Conditioned Risk Gating for Safety-Critical Control under Partial Observability [79.08785366532287]
  We propose a lightweight risk-gated reinforcement learning approximation for risk-sensitive control under partial observability.<n>We evaluate the approach in two safety-critical partially observable domains: automated glucose regulation and safety-constrained navigation.
  arXiv  Detail & Related papers  (2026-05-14T01:23:09Z)
• CMP: Robust Whole-Body Tracking for Loco-Manipulation via Competence Manifold Projection [64.65196237023754]
  Manifold Competence Projection improves robustness against Out-of-Distribution perturbations.<n>We employ a Frame-Wise Safety Scheme that transforms the infinite-horizon safety constraint into a computationally efficient single-step manifold inclusion.<n>Experiments demonstrate that CMP achieves up to a 10-fold survival rate improvement in typical OOD scenarios.
  arXiv  Detail & Related papers  (2026-04-08T18:00:39Z)
• SFCoT: Safer Chain-of-Thought via Active Safety Evaluation and Calibration [9.8659606914942]
  This paper proposes a SaFer Chain-of-Thought (SFCoT) framework, which proactively evaluates and calibrates potentially unsafe reasoning steps in real time.<n> Experimental results demonstrate that SFCoT reduces the attack success rate from $58.97%$ to $12.31%$.
  arXiv  Detail & Related papers  (2026-03-16T15:13:21Z)
• DRAFT: Task Decoupled Latent Reasoning for Agent Safety [59.46137757545185]
  We propose DRAFT (Task Decoupled Latent Reasoning for Agent Safety), a latent reasoning framework that decouples safety judgment into two trainable stages.<n>An Extractor distills the full trajectory into a compact continuous latent draft, and a Reasoner that jointly attends to the draft and the original trajectory to predict safety.<n>DRAFT consistently outperforms strong baselines, improving accuracy from 63.27% (LoRA) to 91.18% averaged over benchmarks.
  arXiv  Detail & Related papers  (2026-02-11T07:45:14Z)
• Interpretable Safety Alignment via SAE-Constructed Low-Rank Subspace Adaptation [13.509767769174422]
  Safety alignment is critical for training large language models to refuse harmful requests.<n>Low-Rank Adaptation (LoRA) consistently underperforms full fine-tuning and reinforcement learning on safety benchmarks.<n>We propose SAILS (Safety Alignment via Interpretable Low-rank Subspace) to address this gap.
  arXiv  Detail & Related papers  (2025-12-29T07:39:49Z)
• GSAE: Graph-Regularized Sparse Autoencoders for Robust LLM Safety Steering [5.124731939041066]
  Large language models (LLMs) face critical safety challenges as they can be manipulated to generate harmful content through adversarial prompts and jailbreak attacks.<n>We introduce Graph-Regularized Sparse Autoencoders (GSAEs), which extends SAEs with a Laplacian smoothness penalty on the neuron co-activation graph.<n>We demonstrate that GSAE enables effective safety steering, assembling features into a weighted set of safety-relevant directions and controlling them with a two-stage gating mechanism.
  arXiv  Detail & Related papers  (2025-12-07T04:46:30Z)
• Boundary-to-Region Supervision for Offline Safe Reinforcement Learning [56.150983204962735]
  Boundary-to-Region (B2R) is a framework that enables asymmetric conditioning through cost signal realignment.<n>B2R redefines CTG as a boundary constraint under a fixed safety budget, unifying the cost distribution of all feasible trajectories.<n> Experimental results show that B2R satisfies safety constraints in 35 out of 38 safety-critical tasks.
  arXiv  Detail & Related papers  (2025-09-30T03:38:20Z)
• SafetyAnalyst: Interpretable, Transparent, and Steerable Safety Moderation for AI Behavior [56.10557932893919]
  We present SafetyAnalyst, a novel AI safety moderation framework.<n>Given an AI behavior, SafetyAnalyst uses chain-of-thought reasoning to analyze its potential consequences.<n>It aggregates effects into a harmfulness score using 28 fully interpretable weight parameters.
  arXiv  Detail & Related papers  (2024-10-22T03:38:37Z)
• Safety Margins for Reinforcement Learning [53.10194953873209]
  We show how to leverage proxy criticality metrics to generate safety margins. We evaluate our approach on learned policies from APE-X and A3C within an Atari environment.
  arXiv  Detail & Related papers  (2023-07-25T16:49:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.