Steering Large Language Models using Conceptors: Improving Addition-Based Activation Engineering

• URL: http://arxiv.org/abs/2410.16314v4
• Date: Mon, 12 May 2025 08:59:12 GMT
• Title: Steering Large Language Models using Conceptors: Improving Addition-Based Activation Engineering
• Authors: Joris Postmus, Steven Abreu,
• Abstract summary: This paper explores activation engineering, where outputs of pre-trained LLMs are controlled by manipulating their activations at inference time.<n>We introduce conceptors - mathematical constructs that represent sets of activation vectors as ellipsoidal regions.<n>Our experiments demonstrate that conceptors outperform traditional methods across multiple steering tasks.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models have transformed AI, yet reliably controlling their outputs remains a challenge. This paper explores activation engineering, where outputs of pre-trained LLMs are controlled by manipulating their activations at inference time. Unlike traditional methods using a single steering vector, we introduce conceptors - mathematical constructs that represent sets of activation vectors as ellipsoidal regions. Conceptors act as soft projection matrices and offer more precise control over complex activation patterns. Our experiments demonstrate that conceptors outperform traditional methods across multiple steering tasks. We further use Boolean operations on conceptors for combined steering goals that empirically outperform additively combining steering vectors on a set of tasks. These results highlight conceptors as a promising tool for more effective steering of LLMs. Our code is available on github.com/jorispos/conceptorsteering.

Related papers

• HyperSteer: Activation Steering at Scale with Hypernetworks [25.6004576064897]
  HyperSteer is a family of hypernetwork-based architectures which are trained end-to-end to generate steering vectors conditioned on the natural language steering prompts.<n>We show that scaling HyperSteer with thousands of steering prompts exceeds the performance of state-of-the-art activation steering methods.
  arXiv  Detail & Related papers  (2025-06-03T18:32:01Z)
• Incentivizing Multimodal Reasoning in Large Models for Direct Robot Manipulation [89.5123417007126]
  We show how to make Large Multimodal Models (LMMs) understand the spatial action space.<n>We also show how to fully exploit the reasoning capacity of LMMs in solving these tasks.<n>Our resulting reasoning model built upon a 7B backbone, named ReasonManip, demonstrates three notable advantages.
  arXiv  Detail & Related papers  (2025-05-19T06:00:14Z)
• ExpertSteer: Intervening in LLMs through Expert Knowledge [71.12193680015622]
  Activation steering offers a promising method to control the generation process of Large Language Models.<n>We propose ExpertSteer, a novel approach that leverages arbitrary specialized expert models to generate steering vectors.<n>We conduct comprehensive experiments using three LLMs on 15 popular benchmarks across four distinct domains.
  arXiv  Detail & Related papers  (2025-05-18T08:55:46Z)
• Steering Risk Preferences in Large Language Models by Aligning Behavioral and Neural Representations [4.029252551781513]
  We propose a principled approach for uncovering steering vectors.<n>We focus on extracting latent risk preferences from large language models.<n>We show that the resulting steering vectors successfully and reliably modulate LLM outputs in line with the targeted behavior.
  arXiv  Detail & Related papers  (2025-05-16T18:23:10Z)
• Interpretable Steering of Large Language Models with Feature Guided Activation Additions [4.496738719682736]
  We introduce Feature Guided Activation Additions (FGAA), a novel activation steering method. By operating in the latent space of a Sparse Autoencoder (SAE), FGAA constructs precise steering vectors. evaluations on Gemma-2-2B and Gemma-2-9B models demonstrate that FGAA outperforms existing steering methods.
  arXiv  Detail & Related papers  (2025-01-17T02:55:23Z)
• Controlling Large Language Models Through Concept Activation Vectors [30.348768212571255]
  We propose Generation with Concept Activation Vector (GCAV), a lightweight model control framework. GCAV ensures accurate control without requiring resource-extensive fine-tuning. Our framework achieves state-of-the-art performance with granular control, allowing for fine-grained adjustments of both the steering layers and the steering magnitudes for individual samples.
  arXiv  Detail & Related papers  (2025-01-10T07:41:48Z)
• Analyzing Fine-tuning Representation Shift for Multimodal LLMs Steering alignment [53.90425382758605]
  We show how fine-tuning alters the internal structure of a model to specialize in new multimodal tasks. Our work sheds light on how multimodal representations evolve through fine-tuning and offers a new perspective for interpreting model adaptation in multimodal tasks.
  arXiv  Detail & Related papers  (2025-01-06T13:37:13Z)
• Semantics-Adaptive Activation Intervention for LLMs via Dynamic Steering Vectors [8.761404991620285]
  Activation intervention has emerged as an effective and economical method to modify the behavior of large language models (LLMs) We propose Semantics-Adaptive Dynamic Intervention (SADI), a novel method that constructs a dynamic steering vector to intervene model activations at inference time. Experimental results show that SADI outperforms established baselines by substantial margins, improving task performance without training.
  arXiv  Detail & Related papers  (2024-10-16T06:58:49Z)
• Improving Instruction-Following in Language Models through Activation Steering [58.876600545898675]
  We derive instruction-specific vector representations from language models and use them to steer models accordingly. We demonstrate how this method can enhance model adherence to constraints such as output format, length, and word inclusion. Our findings demonstrate that activation steering offers a practical and scalable approach for fine-grained control in language generation.
  arXiv  Detail & Related papers  (2024-10-15T08:38:20Z)
• Vector-ICL: In-context Learning with Continuous Vector Representations [75.96920867382859]
  Large language models (LLMs) have shown remarkable in-context learning capabilities on textual data. We explore whether these capabilities can be extended to continuous vectors from diverse domains, obtained from black-box pretrained encoders. In particular, we find that pretraining projectors with general language modeling objectives enables Vector-ICL.
  arXiv  Detail & Related papers  (2024-10-08T02:25:38Z)
• Personalized Steering of Large Language Models: Versatile Steering Vectors Through Bi-directional Preference Optimization [34.05163996072159]
  "steering vectors" are extracted from the activations of human preference data. This work proposes an innovative approach that could produce more effective steering vectors through bi-directional preference optimization. Our method is designed to allow steering vectors to directly influence the generation probability of contrastive human preference data pairs.
  arXiv  Detail & Related papers  (2024-05-28T05:10:40Z)
• Improving Activation Steering in Language Models with Mean-Centring [10.101141087916133]
  We find that taking the average of activations associated with a target dataset, and subtracting the mean of all training activations, results in effective steering vectors. We also apply mean-centring to extract function vectors, more effectively triggering the execution of a range of natural language tasks by a significant margin.
  arXiv  Detail & Related papers  (2023-12-06T18:27:07Z)
• LanguageMPC: Large Language Models as Decision Makers for Autonomous Driving [87.1164964709168]
  This work employs Large Language Models (LLMs) as a decision-making component for complex autonomous driving scenarios. Extensive experiments demonstrate that our proposed method not only consistently surpasses baseline approaches in single-vehicle tasks, but also helps handle complex driving behaviors even multi-vehicle coordination.
  arXiv  Detail & Related papers  (2023-10-04T17:59:49Z)
• Augmented Language Models: a Survey [55.965967655575454]
  This survey reviews works in which language models (LMs) are augmented with reasoning skills and the ability to use tools. We refer to them as Augmented Language Models (ALMs) The missing token objective allows ALMs to learn to reason, use tools, and even act, while still performing standard natural language tasks.
  arXiv  Detail & Related papers  (2023-02-15T18:25:52Z)
• GEM: Group Enhanced Model for Learning Dynamical Control Systems [78.56159072162103]
  We build effective dynamical models that are amenable to sample-based learning. We show that learning the dynamics on a Lie algebra vector space is more effective than learning a direct state transition model. This work sheds light on a connection between learning of dynamics and Lie group properties, which opens doors for new research directions.
  arXiv  Detail & Related papers  (2021-04-07T01:08:18Z)
• Goal-Conditioned End-to-End Visuomotor Control for Versatile Skill Primitives [89.34229413345541]
  We propose a conditioning scheme which avoids pitfalls by learning the controller and its conditioning in an end-to-end manner. Our model predicts complex action sequences based directly on a dynamic image representation of the robot motion. We report significant improvements in task success over representative MPC and IL baselines.
  arXiv  Detail & Related papers  (2020-03-19T15:04:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.