Enhancing Instruction Following of LLMs via Activation Steering with Dynamic Rejection

• URL: http://arxiv.org/abs/2603.06745v1
• Date: Fri, 06 Mar 2026 09:49:32 GMT
• Title: Enhancing Instruction Following of LLMs via Activation Steering with Dynamic Rejection
• Abstract summary: Large Language Models (LLMs) often fail to follow complex user instructions.<n> Activation steering techniques aim to manipulate model internals, but have a potential risk of oversteering.<n>We introduce DIRECTER, a novel steering method that dynamically modulates steering strength by scaling the KV cache without extra dataset.
• Score: 8.819278296219071
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Language Models (LLMs), despite advances in instruction tuning, often fail to follow complex user instructions. Activation steering techniques aim to mitigate this by manipulating model internals, but have a potential risk of oversteering, where excessive emphasis on the instruction degrades task accuracy and overall text quality. To address this, we introduce DIRECTER (Dynamic rejection steering), a novel steering method that dynamically modulates steering strength by scaling the KV cache without extra dataset. DIRECTER couples steering with a plausibility-guided decoding loop, which adaptively adjusts steering strength at each step by comparing the steered output distribution to the original. If the steered output is deemed implausible, steering strength is progressively weakened. This strength modulation is guided by a lightweight, one-time attention sensitivity analysis that ranks layers by their influence on model representations. Extensive evaluations show that DIRECTER significantly enhances instruction-following capabilities across diverse benchmarks, improving accuracy by up to 6.5% over baselines without the common trade-offs in generation quality or task fidelity. The proposed dynamic, plausibility-guided control during activation steering further demonstrates its potential as a general mechanism for mitigating oversteering that is compatible with existing baselines.

Related papers

• Steering Robustness into World Action Models via Mechanistic Interpretability and Optimal Control [3.3394856680250284]
  We study how robustness-relevant perturbations are represented in World Action Models (WAMs) activation space.<n> Comparing activations across successful and unsuccessful rollouts, we find some WAM architectures exhibit low-dimensional linear separability for robustness-critical features.<n>This motivates the use of contrastive activation directions for training-free WAM steering.<n>We also show that local linearity in WAM activation dynamics enables efficient feedback steering via model-based optimal control.
  arXiv  Detail & Related papers  (2026-07-16T12:52:49Z)
• When is Your LLM Steerable? [56.656180566692946]
  Activation steering offers a lightweight approach to control language models' behavior at inference time.<n>Finding the regime and boundaries of successful steering typically requires expensive grid searches.<n>We investigate whether steerability can be predicted from the model's internal states at the beginning of the generation process.
  arXiv  Detail & Related papers  (2026-06-10T02:55:34Z)
• Activation Steering Induces Emergent Misalignment: A More Comprehensive Evaluation [74.17379276939599]
  We show that activation steering can induce broad misalignment, even in the recent Qwen-3.5 series.<n>We characterize properties of AS-induced EM by analyzing key steering-specific factors, including steering magnitude, the low-rank structure of the steering subspace, and the number of epochs during steering-vector construction.
  arXiv  Detail & Related papers  (2026-06-07T15:34:59Z)
• Mitigating Error Amplification in Fast Adversarial Training [58.74042726356826]
  Fast Adversarial Training (FAT) has proven effective in enhancing model robustness by encouraging networks to learn perturbation-invariant representations.<n>FAT often suffers from catastrophic overfitting (CO), where the model overfits to the training attack and fails to generalize to unseen ones.<n>We propose a Distribution-aware Dynamic Guidance (DDG) strategy that dynamically adjusts both the perturbation budget and supervision signal.
  arXiv  Detail & Related papers  (2026-04-27T11:23:18Z)
• ODESteer: A Unified ODE-Based Steering Framework for LLM Alignment [49.68063561145927]
  We propose a unified ordinary differential equations (ODEs)-based theoretical framework for activation steering.<n>We introduce ODESteer, a kind of ODE-based steering guided by barrier functions.<n>Compared to state-of-the-art activation steering methods, ODESteer achieves consistent empirical improvements.
  arXiv  Detail & Related papers  (2026-02-19T17:13:44Z)
• AMPS: Adaptive Modality Preference Steering via Functional Entropy [66.69992693275061]
  We introduce an instance-aware diagnostic metric that quantifies each modality's information contribution and reveals sample-specific susceptibility to steering.<n> Experimental results show that our instance-aware steering outperforms conventional steering in modulating modality preference.
  arXiv  Detail & Related papers  (2026-02-13T02:29:06Z)
• Steering Externalities: Benign Activation Steering Unintentionally Increases Jailbreak Risk for Large Language Models [62.16655896700062]
  Activation steering is a technique to enhance the utility of Large Language Models (LLMs)<n>We show that it unintentionally introduces critical and under-explored safety risks.<n>Experiments reveal that these interventions act as a force multiplier, creating new vulnerabilities to jailbreaks and increasing attack success rates to over 80% on standard benchmarks.
  arXiv  Detail & Related papers  (2026-02-03T12:32:35Z)
• RISER: Orchestrating Latent Reasoning Skills for Adaptive Activation Steering [62.63376387138257]
  We propose a plug-and-play intervention framework that adaptively steers large language models (LLMs) reasoning in activation space.<n>RISER constructs a library of reusable reasoning vectors and employs a lightweight Router to dynamically compose them for each input.<n>The Router is optimized via reinforcement learning under task-level rewards, activating latent cognitive primitives in an emergent and compositional manner.
  arXiv  Detail & Related papers  (2026-01-14T08:04:33Z)
• Dynamically Scaled Activation Steering [3.177576903071419]
  We introduce Dynamically Scaled Activation Steering (DSAS), a method-agnostic steering framework that decouples when to steer from how to steer.<n>DSAS adaptively modulates the strength of existing steering transformations across layers and inputs, intervening strongly only when undesired behavior is detected.
  arXiv  Detail & Related papers  (2025-12-03T10:50:15Z)
• In-Distribution Steering: Balancing Control and Coherence in Language Model Generation [0.0815557531820863]
  We introduce In-Distribution Steering (IDS), a novel method that adapts steering strength based on the input data distribution in representation space.<n>IDS achieves strong accuracy on classification tasks while producing coherent text without collapse, making IDS particularly well suited for real-world applications.
  arXiv  Detail & Related papers  (2025-10-15T08:31:37Z)
• Activation Steering with a Feedback Controller [4.609594868699996]
  Proportional-Integral-Derivative (PID) Steering is a principled framework that leverages the full PID controller for activation steering in large language models.<n>PID Steering consistently outperforms existing approaches, achieving more robust and reliable behavioral control.
  arXiv  Detail & Related papers  (2025-10-05T18:05:28Z)
• KV Cache Steering for Controlling Frozen LLMs [80.50365534625438]
  cache steering is a lightweight method for implicit steering of language models.<n>We apply cache steering to induce chain-of-thought reasoning in small language models.
  arXiv  Detail & Related papers  (2025-07-11T17:59:36Z)
• STU-PID: Steering Token Usage via PID Controller for Efficient Large Language Model Reasoning [0.0]
  Large Language Models employing extended chain-of-thought (CoT) reasoning often suffer from the overthinking phenomenon.<n>We propose STUPID, a novel training-free method that employs a PID controller to dynamically activation modulate steering strength during inference.<n>Our approach combines a chunk-level classifier for detecting redundant reasoning patterns with a PID control mechanism that adaptively adjusts steering intensity based on the predicted redundancy probability.
  arXiv  Detail & Related papers  (2025-06-23T16:47:19Z)
• Instruction Following by Boosting Attention of Large Language Models [11.739148611340964]
  latent steering is a lightweight technique that alters internal activations to guide generation.<n>InstABoost boosts the strength of instruction prompting by altering the model's attention during generation.<n>InstABoost demonstrates superior control success compared to both traditional prompting and latent steering.
  arXiv  Detail & Related papers  (2025-06-16T17:42:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.