CircuitSteer: Geometrically Aligned Multi-Layer Steering via Sparse Autoencoder Circuits

• URL: http://arxiv.org/abs/2608.05732v1
• Date: Thu, 06 Aug 2026 08:17:50 GMT
• Title: CircuitSteer: Geometrically Aligned Multi-Layer Steering via Sparse Autoencoder Circuits
• Abstract summary: Controlling the behavior of large language models (LLMs) is a critical challenge for AI alignment.<n>Existing steering methods rely on fixed single-layer interventions derived from aggregate activation differences.<n>We introduce CircuitSteer, a novel framework that leverages Sparse Autoencoders to identify and manipulate coherent semantic circuits.
• Score: 3.6732711233211677
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Controlling the behavior of large language models (LLMs) remains a critical challenge for AI alignment. Existing steering methods, such as Contrastive Activation Addition (CAA), typically rely on fixed single-layer interventions derived from aggregate activation differences. These methods impose a single intervention across semantically diverse inputs and often fail to sustain consistent behavioral changes across layers, limiting the effectiveness of the steering. In this work, we introduce CircuitSteer, a novel framework that leverages Sparse Autoencoders (SAEs) to identify and manipulate coherent semantic circuits distributed across multiple layers. By constructing a feature flow circuit based on feature co-activation and the geometric alignment of decoder directions, we isolate the specific multi-layer subcircuits responsible for a target behavior. We then synthesize dense steering vectors from these sparse features and apply multi-point interventions to guide the model's internal semantic trajectory. We evaluate CircuitSteer using contrastive examples across a diverse set of tasks, including toxicity, emotion-intensity, sycophancy, and refusal, spanning two model families. Across all models and datasets, CircuitSteer is the only method to consistently produce fluency-preserving interventions; competing methods either sacrifice text quality or lack coverage, failing entirely on complex behaviors like sycophancy and refusal. These results demonstrate that multi-layer circuit steering, enabled by enforcing geometric alignment among selected features, yields strictly more robust and effective behavioral control than static single-point interventions. Code is available at https://github.com/mehrshad-sdtn/CircuitSteer.

Related papers

• Closing the Loop: PID Feedback Control for Interpretable Activation Steering in Symbolic Music Generation [6.897804369133787]
  This paper investigates the mechanistic interpretability of the Multitrack Music Transformer (MMT)<n>We propose a framework for deterministic attribute modulation without retraining to bridge this gap via inference-time activation steering.
  arXiv  Detail & Related papers  (2026-06-17T08:04:13Z)
• Passive Variable Impedance For Shared Control [9.4607769454218]
  We study the stabilization of both variable stiffness in impedance control as well as the arbitration of different controllers.<n>We identify passivity violations in the closed loop system and provide methods to passivate the system.<n>The proposed methods are validated in simulation as well as in real robot experiments on different systems.
  arXiv  Detail & Related papers  (2026-04-22T13:39:39Z)
• Chain of Modality: From Static Fusion to Dynamic Orchestration in Omni-MLLMs [84.3271821505699]
  Chain of Modality (CoM) is an agentic framework that transitions multimodal fusion from passive concatenation to dynamic orchestration.<n> operating in either a training-free or a data-efficient SFT setting, CoM achieves robust and consistent generalization across diverse benchmarks.
  arXiv  Detail & Related papers  (2026-04-16T01:21:14Z)
• What Drives Representation Steering? A Mechanistic Case Study on Steering Refusal [53.189667624047416]
  We show that different steering methodologies leverage functionally interchangeable circuits when applied at the same layer.<n>We show that steering vectors can be sparsified by up to 90-99% while retaining most performance.
  arXiv  Detail & Related papers  (2026-04-09T17:57:14Z)
• Primary-Fine Decoupling for Action Generation in Robotic Imitation [91.2899765310853]
  Multi-modal distribution in robotic manipulation action sequences poses critical challenges for imitation learning.<n>We propose Primary-Fine Decoupling for Action Generation (PF-DAG), a two-stage framework that decouples coarse action consistency from fine-grained variations.<n>PF-DAG outperforms state-of-the-art baselines across 56 tasks from Adroit, DexArt, and MetaWorld benchmarks.
  arXiv  Detail & Related papers  (2026-02-25T08:36:45Z)
• Controllable and explainable personality sliders for LLMs at inference time [1.3688381983244782]
  We propose a modular framework for continuous, multi-dimensional personality control.<n>Our key innovation is Sequential Adaptive Steering (SAS), a method that transforms steering vectors by training subsequent probes on the residual stream shifted by prior interventions.<n>We validate our framework on the Big Five personality traits, demonstrating that it outperforms naive baselines in both goal adherence and coherence.
  arXiv  Detail & Related papers  (2026-02-10T08:16:59Z)
• Beyond Linear Steering: Unified Multi-Attribute Control for Language Models [8.223479896191145]
  We introduce K-Steering, a unified and flexible approach that trains a single non-linear multi-label classifier on hidden activations.<n>This avoids linearity assumptions, removes the need for storing and tuning separate vectors attribute, and allows dynamic composition of behaviors without retraining.<n> Empirical results across 3 model families, validated by both activation-based classifiers and LLM-based judges, demonstrate that K-Steering outperforms strong baselines in accurately steering multiple behaviors.
  arXiv  Detail & Related papers  (2025-05-30T12:41:19Z)
• Position-aware Automatic Circuit Discovery [59.64762573617173]
  We identify a gap in existing circuit discovery methods, treating model components as equally relevant across input positions.<n>We propose two improvements to incorporate positionality into circuits, even on tasks containing variable-length examples.<n>Our approach enables fully automated discovery of position-sensitive circuits, yielding better trade-offs between circuit size and faithfulness compared to prior work.
  arXiv  Detail & Related papers  (2025-02-07T00:18:20Z)
• Decoupled Multi-task Learning with Cyclical Self-Regulation for Face Parsing [71.19528222206088]
  We propose a novel Decoupled Multi-task Learning with Cyclical Self-Regulation for face parsing. Specifically, DML-CSR designs a multi-task model which comprises face parsing, binary edge, and category edge detection. Our method achieves the new state-of-the-art performance on the Helen, CelebA-HQ, and LapaMask datasets.
  arXiv  Detail & Related papers  (2022-03-28T02:12:30Z)
• Unsupervised Controllable Generation with Self-Training [90.04287577605723]
  controllable generation with GANs remains a challenging research problem. We propose an unsupervised framework to learn a distribution of latent codes that control the generator through self-training. Our framework exhibits better disentanglement compared to other variants such as the variational autoencoder.
  arXiv  Detail & Related papers  (2020-07-17T21:50:35Z)
• A Framework for Automatic Behavior Generation in Multi-Function Swarms [1.290382979353427]
  A framework for automatic behavior generation in multi-function swarms is proposed. The framework is tested on a scenario with three simultaneous tasks. The effect of noise on the behavior characteristics in MAP-elites is investigated.
  arXiv  Detail & Related papers  (2020-07-11T20:50:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.