Measuring Activation Control in Large Language Models

• URL: http://arxiv.org/abs/2608.21664v1
• Date: Fri, 21 Aug 2026 22:12:14 GMT
• Title: Measuring Activation Control in Large Language Models
• Abstract summary: We quantify the extent to which models can modulate their residual stream via natural-language instruction.<n>Across model families and capability levels, we find that most LLMs can control the direction and magnitude of their residual stream activations.<n>These results suggest that control over the activation space itself could become a confound for monitoring as introspective capabilities increase.
• Score: 0.5171493078033068
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Safe deployment of increasingly capable models will likely come to rely on latent-space monitoring as a complement to behavioral evaluations, especially when evaluation-aware models exhibit scheming or deception. However, if models can also control their own activations, deception could extend into the latent space itself. With this in mind, we introduce the Activation Controllability Benchmark to quantify the extent to which models can modulate their residual stream via natural-language instruction. Across model families and capability levels, we find that most LLMs can control the direction and magnitude of their residual stream activations with some degree of temporal resolution, though performance varies considerably across models. In simple tasks, this level of control can evade activation-based monitoring methods (including linear probes, natural language autoencoders, activation oracles, and the Jacobian lens), albeit imperfectly. These results suggest that control over the activation space itself could become a confound for monitoring as introspective capabilities increase; therefore, we recommend that frontier labs and evaluators track activation controllability in future models.

Related papers

• Steering Robustness into World Action Models via Mechanistic Interpretability and Optimal Control [3.3394856680250284]
  We study how robustness-relevant perturbations are represented in World Action Models (WAMs) activation space.<n> Comparing activations across successful and unsuccessful rollouts, we find some WAM architectures exhibit low-dimensional linear separability for robustness-critical features.<n>This motivates the use of contrastive activation directions for training-free WAM steering.<n>We also show that local linearity in WAM activation dynamics enables efficient feedback steering via model-based optimal control.
  arXiv  Detail & Related papers  (2026-07-16T12:52:49Z)
• CIAware-Bench: Benchmarking Control Intervention Awareness Across Frontier LLMs [100.38986535324284]
  We introduce textbfCIAware-Bench, a benchmark for measuring textbfcontrol textbfintervention (CI) awareness across frontier models.<n>CIAware-Bench tests whether models can distinguish their own trajectories from those modified by a control intervention.
  arXiv  Detail & Related papers  (2026-06-09T16:24:16Z)
• Activation Steering Induces Emergent Misalignment: A More Comprehensive Evaluation [74.17379276939599]
  We show that activation steering can induce broad misalignment, even in the recent Qwen-3.5 series.<n>We characterize properties of AS-induced EM by analyzing key steering-specific factors, including steering magnitude, the low-rank structure of the steering subspace, and the number of epochs during steering-vector construction.
  arXiv  Detail & Related papers  (2026-06-07T15:34:59Z)
• Agentic Control in Variational Language Models [0.0]
  We study whether a variational language model can support a minimal and measurable form of agentic control grounded in its own internal evidence.<n>Our model combines local variational hidden computation (EVE), a homeostatic latent regulator, structurally aware checkpoint retention and a calibrated uncertainty-aware controller operating on top of the retained model.
  arXiv  Detail & Related papers  (2026-04-14T09:47:53Z)
• Gated Condition Injection without Multimodal Attention: Towards Controllable Linear-Attention Transformers [69.60693631859411]
  This paper explores controllable diffusion models built upon linear attention architectures.<n>We propose a novel controllable diffusion framework tailored for linear attention backbones like SANA.<n>Our approach achieves state-of-the-art controllable generation performance based on linear-attention models.
  arXiv  Detail & Related papers  (2026-03-29T12:31:34Z)
• Large language models require a new form of oversight: capability-based monitoring [10.382163755118713]
  Large language models (LLMs) in healthcare have been accompanied by scrutiny of their oversight.<n>We propose a new organizing principle guiding generalist LLM monitoring that is scalable and grounded in how these models are developed and used in practice: capability-based monitoring.<n>We describe considerations for developers, organizational leaders, and professional societies for implementing a capability-based monitoring approach.
  arXiv  Detail & Related papers  (2025-11-05T01:20:28Z)
• ScaleWeaver: Weaving Efficient Controllable T2I Generation with Multi-Scale Reference Attention [86.93601565563954]
  ScaleWeaver is a framework designed to achieve high-fidelity, controllable generation upon advanced visual autoregressive( VAR) models.<n>The proposed Reference Attention module discards the unnecessary attention from image$rightarrow$condition, reducing computational cost.<n>Experiments show that ScaleWeaver delivers high-quality generation and precise control while attaining superior efficiency over diffusion-based methods.
  arXiv  Detail & Related papers  (2025-10-16T17:00:59Z)
• Improving Reasoning Performance in Large Language Models via Representation Engineering [2.0099933815960256]
  We propose a representation engineering approach for large language models (LLMs)<n>Model activations are read from the residual stream of an LLM when processing a reasoning task.<n>We show that an LLM can, to a certain degree, be controlled to improve its perceived reasoning ability by modulating activations.
  arXiv  Detail & Related papers  (2025-04-28T04:58:43Z)
• Controlling Large Language Models Through Concept Activation Vectors [30.348768212571255]
  We propose Generation with Concept Activation Vector (GCAV), a lightweight model control framework.<n>GCAV ensures accurate control without requiring resource-extensive fine-tuning.<n>Our framework achieves state-of-the-art performance with granular control, allowing for fine-grained adjustments of both the steering layers and the steering magnitudes for individual samples.
  arXiv  Detail & Related papers  (2025-01-10T07:41:48Z)
• Model-Based Reinforcement Learning with Isolated Imaginations [61.67183143982074]
  We propose Iso-Dream++, a model-based reinforcement learning approach. We perform policy optimization based on the decoupled latent imaginations. This enables long-horizon visuomotor control tasks to benefit from isolating mixed dynamics sources in the wild.
  arXiv  Detail & Related papers  (2023-03-27T02:55:56Z)
• Predictive Coding for Locally-Linear Control [92.35650774524399]
  High-dimensional observations and unknown dynamics are major challenges when applying optimal control to many real-world decision making tasks. The Learning Controllable Embedding (LCE) framework addresses these challenges by embedding the observations into a lower dimensional latent space. We show theoretically that explicit next-observation prediction can be replaced with predictive coding.
  arXiv  Detail & Related papers  (2020-03-02T18:20:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.