Steerability of Instrumental-Convergence Tendencies in LLMs

• URL: http://arxiv.org/abs/2601.01584v2
• Date: Tue, 06 Jan 2026 16:11:57 GMT
• Title: Steerability of Instrumental-Convergence Tendencies in LLMs
• Abstract summary: We examine two properties of AI systems: capability (what a system can do) and steerability.<n>We distinguish between authorized steerability ( reliably reaching intended behaviors) and unauthorized steerability.<n>This distinction highlights a fundamental safety--security dilemma of AI models.
• Score: 0.5922488908114023
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We examine two properties of AI systems: capability (what a system can do) and steerability (how reliably one can shift behavior toward intended outcomes). A central question is whether capability growth reduces steerability and risks control collapse. We also distinguish between authorized steerability (builders reliably reaching intended behaviors) and unauthorized steerability (attackers eliciting disallowed behaviors). This distinction highlights a fundamental safety--security dilemma of AI models: safety requires high steerability to enforce control (e.g., stop/refuse), while security requires low steerability for malicious actors to elicit harmful behaviors. This tension presents a significant challenge for open-weight models, which currently exhibit high steerability via common techniques like fine-tuning or adversarial attacks. Using Qwen3 and InstrumentalEval, we find that a short anti-instrumental prompt suffix sharply reduces the measured convergence rate (e.g., shutdown avoidance, self-replication). For Qwen3-30B Instruct, the convergence rate drops from 81.69% under a pro-instrumental suffix to 2.82% under an anti-instrumental suffix. Under anti-instrumental prompting, larger aligned models show lower convergence rates than smaller ones (Instruct: 2.82% vs. 4.23%; Thinking: 4.23% vs. 9.86%). Code is available at github.com/j-hoscilowicz/instrumental_steering.

Related papers

• Asymmetric Collapse in Model Merging: When Refusal Over- writes Recognition [4.800831341795478]
  We use two Gemma-3-1B-IT finetunes on two complementary safety objectives: CARES harm-level classification and WildJailbreak adversarial refusal.<n>We merge the two fine-tunes using Linear, SLERP, TIES, and DARE-TIES, and evaluate the merged models on classification accuracy, attack resistance, and benign compliance.
  arXiv  Detail & Related papers  (2026-07-26T19:10:34Z)
• Evaluation Awareness Is Not One Capability: Evidence from Open Language Models [1.758143872501506]
  Safety benchmarks assume that test-condition behavior predicts deployment behavior, an assumption that fails if models detect evaluation cues and adapt.<n>This opens a gap between benchmark performance and deployment behavior.<n>We characterize this evaluation awareness through eight experiments across 37 open-weight models and seven families.
  arXiv  Detail & Related papers  (2026-06-22T16:48:43Z)
• The Geometry of Refusal: Linear Instability in Safety-Aligned LLMs [1.4323566945483497]
  We introduce Contrastive Logit Steering ( CLS), a zero-optimization framework that isolates the "refusal direction"<n> CLS operates directly on the output distribution, serving as a diagnostic probe for alignment.<n>Our experiments on 7 model families reveal that safety implementation is architecturally deterministic.
  arXiv  Detail & Related papers  (2026-06-21T22:04:48Z)
• Persona Non Grata: Single-Method Safety Evaluation Is Incomplete for Persona-Imbued LLMs [5.478971182058342]
  Safety evaluations almost always study prompt-based personas alone.<n>We show this is incomplete: prompting and activation steering expose *different*, architecture-dependent vulnerability profiles.
  arXiv  Detail & Related papers  (2026-04-13T07:34:02Z)
• Claw-Eval: Toward Trustworthy Evaluation of Autonomous Agents [66.97968363332465]
  We introduce Claw-Eval, an end-to-end evaluation suite addressing all three gaps in agent benchmarks.<n>Claw-Eval comprises 300 human-verified tasks spanning 9 categories across three groups.<n>Every agent action is recorded through three independent evidence channels.
  arXiv  Detail & Related papers  (2026-04-07T17:43:18Z)
• Steering Safely or Off a Cliff? Rethinking Specificity and Robustness in Inference-Time Interventions [2.977664945581083]
  We show that while steering achieves high efficacy and largely maintains general and control specificity, it consistently fails to preserve robustness specificity.<n>Our work provides the first systematic evaluation of specificity in model steering, showing that standard efficacy and specificity checks are insufficient.
  arXiv  Detail & Related papers  (2026-02-05T23:14:05Z)
• Steering Externalities: Benign Activation Steering Unintentionally Increases Jailbreak Risk for Large Language Models [62.16655896700062]
  Activation steering is a technique to enhance the utility of Large Language Models (LLMs)<n>We show that it unintentionally introduces critical and under-explored safety risks.<n>Experiments reveal that these interventions act as a force multiplier, creating new vulnerabilities to jailbreaks and increasing attack success rates to over 80% on standard benchmarks.
  arXiv  Detail & Related papers  (2026-02-03T12:32:35Z)
• Agentic Uncertainty Quantification [76.94013626702183]
  We propose a unified Dual-Process Agentic UQ (AUQ) framework that transforms verbalized uncertainty into active, bi-directional control signals.<n>Our architecture comprises two complementary mechanisms: System 1 (Uncertainty-Aware Memory, UAM), which implicitly propagates verbalized confidence and semantic explanations to prevent blind decision-making; and System 2 (Uncertainty-Aware Reflection, UAR), which utilizes these explanations as rational cues to trigger targeted inference-time resolution only when necessary.
  arXiv  Detail & Related papers  (2026-01-22T07:16:26Z)
• GSAE: Graph-Regularized Sparse Autoencoders for Robust LLM Safety Steering [5.124731939041066]
  Large language models (LLMs) face critical safety challenges as they can be manipulated to generate harmful content through adversarial prompts and jailbreak attacks.<n>We introduce Graph-Regularized Sparse Autoencoders (GSAEs), which extends SAEs with a Laplacian smoothness penalty on the neuron co-activation graph.<n>We demonstrate that GSAE enables effective safety steering, assembling features into a weighted set of safety-relevant directions and controlling them with a two-stage gating mechanism.
  arXiv  Detail & Related papers  (2025-12-07T04:46:30Z)
• Death by a Thousand Prompts: Open Model Vulnerability Analysis [0.06213771671016099]
  Open-weight models provide researchers and developers with foundations for diverse downstream applications.<n>We tested the safety and security postures of eight open-weight large language models (LLMs) to identify vulnerabilities that may impact subsequent fine-tuning and deployment.<n>Our findings reveal pervasive vulnerabilities across all tested models, with multi-turn attacks achieving success rates between 25.86% and 92.78%.
  arXiv  Detail & Related papers  (2025-11-05T07:22:24Z)
• Impatient Users Confuse AI Agents: High-fidelity Simulations of Human Traits for Testing Agents [58.00130492861884]
  TraitBasis is a lightweight, model-agnostic method for systematically stress testing AI agents.<n>TraitBasis learns directions in activation space corresponding to steerable user traits.<n>We observe on average a 2%-30% performance degradation on $tau$-Trait across frontier models.
  arXiv  Detail & Related papers  (2025-10-06T05:03:57Z)
• Does higher interpretability imply better utility? A Pairwise Analysis on Sparse Autoencoders [63.544453925182005]
  We train 90 SAEs across three language models and evaluate their interpretability and steering utility.<n>Our analysis reveals only a relatively weak positive association (tau b approx 0.298), indicating that interpretability is an insufficient proxy for steering performance.<n>We propose a novel selection criterion called Delta Token Confidence, which measures how much amplifying a feature changes the next token distribution.
  arXiv  Detail & Related papers  (2025-10-04T04:14:50Z)
• DiffuGuard: How Intrinsic Safety is Lost and Found in Diffusion Large Language Models [50.21378052667732]
  We conduct an in-depth analysis of dLLM vulnerabilities to jailbreak attacks across two distinct dimensions: intra-step and inter-step dynamics.<n>We propose DiffuGuard, a training-free defense framework that addresses vulnerabilities through a dual-stage approach.
  arXiv  Detail & Related papers  (2025-09-29T05:17:10Z)
• AdvChain: Adversarial Chain-of-Thought Tuning for Robust Safety Alignment of Large Reasoning Models [62.70575022567081]
  We propose AdvChain, an alignment paradigm that teaches models dynamic self-correction through adversarial CoT tuning.<n>Our work establishes a new direction for building more robust and reliable reasoning models.
  arXiv  Detail & Related papers  (2025-09-29T04:27:23Z)
• The Rogue Scalpel: Activation Steering Compromises LLM Safety [11.402179030703188]
  Activation steering is a technique for controlling LLM behavior by adding semantically meaningful vectors directly into a model's hidden states during inference.<n>We demonstrate the opposite: steering systematically breaks model alignment safeguards, making it comply with harmful requests.
  arXiv  Detail & Related papers  (2025-09-26T08:49:47Z)
• AdaSteer: Your Aligned LLM is Inherently an Adaptive Jailbreak Defender [99.3105257001476]
  We propose AdaSteer, an adaptive activation steering method that adjusts model behavior based on input characteristics.<n>AdaSteer steers input representations along both the Rejection Direction (RD) and Harmfulness Direction (HD)<n>Our results highlight the potential of interpretable model internals for real-time, flexible safety enforcement in LLMs.
  arXiv  Detail & Related papers  (2025-04-13T07:39:17Z)
• Attacker Control and Bug Prioritization [5.639904484784127]
  Control of an attacker over a vulnerability's parameters is an often overlooked factor of exploitability.<n>We show that taint as well as straightforward qualitative and quantitative notions of control are not enough to effectively differentiate vulnerabilities.<n>We propose to focus analysis on feasible value sets, which we call domains of control, in order to better take into account threat models and expert insight.
  arXiv  Detail & Related papers  (2025-01-29T16:27:43Z)
• Steering Language Model Refusal with Sparse Autoencoders [16.304363931580273]
  This work uncovers a tension between SAE steering-based safety improvements and general model capabilities.<n>Our findings reveal important open questions about the nature of safety-relevant features in language models.
  arXiv  Detail & Related papers  (2024-11-18T05:47:02Z)
• Criticality and Safety Margins for Reinforcement Learning [53.10194953873209]
  We seek to define a criticality framework with both a quantifiable ground truth and a clear significance to users.<n>We introduce true criticality as the expected drop in reward when an agent deviates from its policy for n consecutive random actions.<n>We also introduce the concept of proxy criticality, a low-overhead metric that has a statistically monotonic relationship to true criticality.
  arXiv  Detail & Related papers  (2024-09-26T21:00:45Z)
• Steering Without Side Effects: Improving Post-Deployment Control of Language Models [61.99293520621248]
  Language models (LMs) have been shown to behave unexpectedly post-deployment. We present KL-then-steer (KTS), a technique that decreases the side effects of steering while retaining its benefits. Our best method prevents 44% of jailbreak attacks compared to the original Llama-2-chat-7B model.
  arXiv  Detail & Related papers  (2024-06-21T01:37:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.