Activation Steering Induces Emergent Misalignment: A More Comprehensive Evaluation

• URL: http://arxiv.org/abs/2606.08682v1
• Date: Sun, 07 Jun 2026 15:34:59 GMT
• Title: Activation Steering Induces Emergent Misalignment: A More Comprehensive Evaluation
• Abstract summary: We show that activation steering can induce broad misalignment, even in the recent Qwen-3.5 series.<n>We characterize properties of AS-induced EM by analyzing key steering-specific factors, including steering magnitude, the low-rank structure of the steering subspace, and the number of epochs during steering-vector construction.
• Score: 74.17379276939599
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Activation steering has emerged as a popular inference-time technique for modulating the behavior of large language models (LLMs). By constructing a steering vector from examples of a target behavior and injecting it into intermediate activations during inference, activation steering enables flexible behavioral control while avoiding the permanent parameter updates required by finetuning. Meanwhile, recent work has identified emergent misalignment (EM) as a significant safety concern, wherein models finetuned on unsafe examples from a narrow task may unexpectedly generalize to broadly unsafe behavior on unrelated tasks. Although finetuning-induced EM has been extensively studied, whether activation steering can induce EM remains comparatively under-explored, despite its increasing use as a model-control technique. In this paper, we present a comprehensive study of activation-steering-induced emergent misalignment, substantially expanding the evaluation scope beyond existing pioneering work. First, we show that activation steering can induce broad misalignment, even in the recent Qwen-3.5 series. Moreover, activation-steered models produce harmful responses with stronger semantic relevance and higher coherence than their finetuned counterparts, making the resulting misalignment potentially more harmful. Second, we characterize properties of AS-induced EM by analyzing key steering-specific factors, including steering magnitude, the low-rank structure of the steering subspace, and the number of epochs during steering-vector construction. Third, we evaluate the robustness and sensitivity of AS-induced EM across diverse model families, model scales, target tasks, and intervention layers. Our findings reveal activation steering as a significant yet under-examined source of emergent misalignment and provide an activation-space perspective for understanding the mechanisms and safety risks of EM.

Related papers

• Looking in the Mirror: Introspecting Side-Effect Misalignments Induced by Fine-Tuning [17.810096531720927]
  Fine-tuning enables a source model to acquire desired capabilities and behaviors in a target domain.<n>Fine-tuning can also degrade alignment properties that were present in the source model.<n>Introspection adapters describe behavioral changes induced by fine-tuning.
  arXiv  Detail & Related papers  (2026-08-05T01:47:07Z)
• Sensitivity Shaping for Latent Modeling [24.352929199796236]
  We introduce support-conditioned control-sensitivity regularization, which promotes sensitive local response to control input changes in learned dynamics.<n>Experiments in vision-based obstacle avoidance, manipulation, and real-robot navigation show improved OOD detection and safer closed-loop planning.
  arXiv  Detail & Related papers  (2026-06-12T16:01:50Z)
• Minimizing Collateral Damage in Activation Steering [26.274685653829792]
  Activation steering is a method for controlling Large Language Model (LLM) behavior by intervening in its internal representations to increase the alignment with a specific target feature direction.<n>Standard interventions, such as vector addition, often cause unintended changes in the alignment of activations along other non-target feature directions.<n>In this work, we provide a mathematical formalization of collateral damage and introduce a principled framework that models steering as a constrained optimization problem.
  arXiv  Detail & Related papers  (2026-05-01T23:52:54Z)
• Mitigating Error Amplification in Fast Adversarial Training [58.74042726356826]
  Fast Adversarial Training (FAT) has proven effective in enhancing model robustness by encouraging networks to learn perturbation-invariant representations.<n>FAT often suffers from catastrophic overfitting (CO), where the model overfits to the training attack and fails to generalize to unseen ones.<n>We propose a Distribution-aware Dynamic Guidance (DDG) strategy that dynamically adjusts both the perturbation budget and supervision signal.
  arXiv  Detail & Related papers  (2026-04-27T11:23:18Z)
• Principled Steering via Null-space Projection for Jailbreak Defense in Vision-Language Models [71.11171576590134]
  Vision-language models can be easily induced by visual jailbreak attacks to generate harmful content.<n>We propose NullSteer, a null-space projected activation defense framework.<n>We show that NullSteer significantly reduces harmful outputs under various jailbreak attacks.
  arXiv  Detail & Related papers  (2026-03-23T15:23:23Z)
• Enhancing Instruction Following of LLMs via Activation Steering with Dynamic Rejection [8.819278296219071]
  Large Language Models (LLMs) often fail to follow complex user instructions.<n> Activation steering techniques aim to manipulate model internals, but have a potential risk of oversteering.<n>We introduce DIRECTER, a novel steering method that dynamically modulates steering strength by scaling the KV cache without extra dataset.
  arXiv  Detail & Related papers  (2026-03-06T09:49:32Z)
• Steering Externalities: Benign Activation Steering Unintentionally Increases Jailbreak Risk for Large Language Models [62.16655896700062]
  Activation steering is a technique to enhance the utility of Large Language Models (LLMs)<n>We show that it unintentionally introduces critical and under-explored safety risks.<n>Experiments reveal that these interventions act as a force multiplier, creating new vulnerabilities to jailbreaks and increasing attack success rates to over 80% on standard benchmarks.
  arXiv  Detail & Related papers  (2026-02-03T12:32:35Z)
• Mechanistic Indicators of Steering Effectiveness in Large Language Models [3.635648354808971]
  Activation-based steering enables Large Language Models to exhibit targeted behaviors by intervening on intermediate activations without retraining.<n>Despite its widespread use, the mechanistic factors that govern when steering succeeds or fails remain poorly understood.<n>We investigate whether the reliability of steering can be diagnosed using internal model signals.
  arXiv  Detail & Related papers  (2026-02-02T06:56:22Z)
• From Passive Metric to Active Signal: The Evolving Role of Uncertainty Quantification in Large Language Models [77.04403907729738]
  This survey charts the evolution of uncertainty from a passive diagnostic metric to an active control signal guiding real-time model behavior.<n>We demonstrate how uncertainty is leveraged as an active control signal across three frontiers.<n>This survey argues that mastering the new trend of uncertainty is essential for building the next generation of scalable, reliable, and trustworthy AI.
  arXiv  Detail & Related papers  (2026-01-22T06:21:31Z)
• Steering Latent Traits, Not Learned Facts: An Empirical Study of Activation Control Limits [0.0]
  Large language models (LLMs) require precise behavior control for safe and effective deployment across diverse applications.<n>We focus on the question of how steering effectiveness varies across different behavior types and whether the nature of target behaviors can predict steering success.
  arXiv  Detail & Related papers  (2025-11-23T04:28:41Z)
• Beyond Prompt Engineering: Robust Behavior Control in LLMs via Steering Target Atoms [71.85633762642125]
  The vast number of parameters in models often results in highly intertwined internal representations.<n>Recent research has explored the use of sparse autoencoders (SAE) to disentangle knowledge in high-dimensional spaces for steering.<n>We propose Steering Target Atoms (STA), a novel method that isolates and manipulates disentangled knowledge components to enhance safety.
  arXiv  Detail & Related papers  (2025-05-23T17:59:18Z)
• Patterns and Mechanisms of Contrastive Activation Engineering [0.374490703387131]
  CAE has the potential to introduce a new paradigm of flexible, task-specific behavior tuning.<n>We analyze the performance of CAE in in in-distribution, out-of-distribution settings, evaluate drawbacks, and begin to develop comprehensive guidelines for its effective deployment.
  arXiv  Detail & Related papers  (2025-05-06T05:15:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.