UniSteer: Text-Guided Flow Matching in Activation Space for Versatile LLM Steering

• URL: http://arxiv.org/abs/2605.30076v1
• Date: Thu, 28 May 2026 15:23:29 GMT
• Title: UniSteer: Text-Guided Flow Matching in Activation Space for Versatile LLM Steering
• Abstract summary: Activation-based control has emerged as an effective paradigm for controlling behaviors such as persona and style.<n>We propose UniSteer, a text-guided activation flow matching model that learns a conditional distribution over residual-stream activations.<n>We show that UniSteer provides a unified interface across behavioral control, truthfulness steering, fine-grained concept steering, multi-constraint instruction following, and activation-space classification.
• Score: 42.82337069202406
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Activation-based control steers large language models (LLMs) by intervening on their internal representations during inference, and has emerged as an effective paradigm for controlling behaviors such as persona and style. However, existing methods often rely on fixed steering directions or task-specific intervention modules, making them difficult to adapt to fine-grained concepts and compositional constraints. We propose UniSteer, a text-guided activation flow matching model that learns a conditional distribution over residual-stream activations from natural-language conditions. Instead of fitting a separate intervention for each target behavior, UniSteer learns a universal conditional velocity field in activation space. At inference time, UniSteer performs flow inversion by partially transporting a source activation toward a latent state and regenerating it under a target textual condition before injecting it back into the frozen LLM. The same conditional model supports activation-space classification by selecting the textual label with the lowest reconstruction energy. Experiments on three target LLMs show that UniSteer provides a unified interface across behavioral control, truthfulness steering, fine-grained concept steering, multi-constraint instruction following, and activation-space classification.

Related papers

• SCRIPT: Scalable Diffusion Policy with Multi-stage Training for Language-driven Physics-based Humanoid Control [72.5651722107621]
  SCRIPT is a scalable diffusion policy with a multi-stage training framework for language-driven physics-based humanoid control.<n>The core of SCRIPT is a Joint Action-State-Text Diffusion Transformer (JAST-DiT), which represents actions, physical states, and text as dedicated token streams.<n>To stabilize autore control, we introduce a nonlinear history conditioning mechanism, which preserves the dense recent context and samples increasingly sparse cues from long-term history.
  arXiv  Detail & Related papers  (2026-05-21T14:17:21Z)
• Steering Code LLMs with Activation Directions for Language and Library Control [10.49462610276906]
  Code LLMs often default to particular programming languages and libraries under neutral prompts.<n>We investigate whether these preferences are encoded as approximately linear directions in activation space that can be manipulated at inference time.
  arXiv  Detail & Related papers  (2026-03-24T18:12:05Z)
• FreeAct: Freeing Activations for LLM Quantization [89.97086263978058]
  Quantization is pivotal for mitigating the significant memory and computational overhead of Large Language Models.<n>FreeAct is a novel quantization framework that relaxes the static one-to-one constraint to accommodate dynamic activation disparities.<n>Experiments across dLLMs and MLLMs demonstrate that FreeAct significantly outperforms baselines, up to 5.3% performance improvement.
  arXiv  Detail & Related papers  (2026-03-02T12:02:17Z)
• ODESteer: A Unified ODE-Based Steering Framework for LLM Alignment [49.68063561145927]
  We propose a unified ordinary differential equations (ODEs)-based theoretical framework for activation steering.<n>We introduce ODESteer, a kind of ODE-based steering guided by barrier functions.<n>Compared to state-of-the-art activation steering methods, ODESteer achieves consistent empirical improvements.
  arXiv  Detail & Related papers  (2026-02-19T17:13:44Z)
• Fine-Grained Activation Steering: Steering Less, Achieving More [33.680685349571]
  Activation steering has emerged as a cost-effective paradigm for modifying large language model (LLM) behaviors.<n>We show that block-level activations are inherently heterogeneous, entangling beneficial, irrelevant, and harmful features.<n>We propose AUSteer, a simple and efficient method that operates at a finer granularity of the AU level.
  arXiv  Detail & Related papers  (2026-02-04T11:03:01Z)
• Steering Language Models Before They Speak: Logit-Level Interventions [9.055997973281919]
  We propose a training-free inference-time logit intervention for controllable generation.<n>Our results show that statistically grounded logit steering can achieve large, consistent, and multi-task control gains.
  arXiv  Detail & Related papers  (2026-01-16T03:00:33Z)
• GrAInS: Gradient-based Attribution for Inference-Time Steering of LLMs and VLMs [56.93583799109029]
  GrAInS is an inference-time steering approach that operates across both language-only and vision-language models and tasks.<n>During inference, GrAInS hidden activations at transformer layers guided by token-level attribution signals, and normalizes activations to preserve representational scale.<n>It consistently outperforms both fine-tuning and existing steering baselines.
  arXiv  Detail & Related papers  (2025-07-24T02:34:13Z)
• LANGTRAJ: Diffusion Model and Dataset for Language-Conditioned Trajectory Simulation [102.1527101235251]
  LangTraj is a language-conditioned scene-diffusion model that simulates the joint behavior of all agents in traffic scenarios.<n>By conditioning on natural language inputs, LangTraj provides flexible and intuitive control over interactive behaviors.<n>LangTraj demonstrates strong performance in realism, language controllability, and language-conditioned safety-critical simulation.
  arXiv  Detail & Related papers  (2025-04-15T17:14:06Z)
• LF-Steering: Latent Feature Activation Steering for Enhancing Semantic Consistency in Large Language Models [16.37602070339033]
  Large Language Models (LLMs) often generate inconsistent responses when prompted with semantically equivalent paraphrased inputs.<n>We propose LF-Steering, a novel activation steering approach to precisely identify latent feature representations responsible for semantic inconsistency.<n>Our method maps the hidden states of the relevant transformer layer into a sparsely activated, high-dimensional feature space based on a sparse autoencoder.
  arXiv  Detail & Related papers  (2025-01-19T13:06:51Z)
• Linearly Controlled Language Generation with Performative Guarantees [4.447467536572626]
  We use a common model of concept semantics as linearly represented in an LM's latent space.<n>We take the view that natural language generation traces a trajectory in this continuous semantic space.<n>We propose a lightweight, gradient-free intervention that dynamically steers trajectories away from regions corresponding to undesired meanings.
  arXiv  Detail & Related papers  (2024-05-24T11:30:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.