Fast & Faithful Function Vectors

• URL: http://arxiv.org/abs/2606.05079v1
• Date: Wed, 03 Jun 2026 16:36:15 GMT
• Title: Fast & Faithful Function Vectors
• Abstract summary: Function vectors (FVs) are task representations elicited during in-context learning that can be used to steer Large Language Models (LLMs)<n>We study the impact of varying FV definitions for instructions along two degrees of freedom: attention head selection and steering.
• Score: 24.484350585583233
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Function vectors (FVs) are task representations elicited during in-context learning that can be used to steer Large Language Models (LLMs). However, design choices in their formulation remain underexplored. In this work, we study the impact of varying FV definitions for instructions along two degrees of freedom: attention head selection and steering. For head selection, using gradient-based attributions with Layer-wise Relevance Propagation (LRP) substantially improves efficiency as well as accuracy. For FV steering, applying it in a distributed manner yields a higher accuracy compared to simple aggregation. Our code is publicly available.

Related papers

• VSPO: Vector-Steered Policy Optimization for Behavioral Control [30.80095775190934]
  We introduce Vector-Steered Policy optimization (VSPO) for language models.<n>VSPO employs a steering vector associated with the target behavior to control the behavior intensity of the generated rollouts.<n>We show that VSPO consistently improves the control along target behavior while maintaining or improving task accuracy.
  arXiv  Detail & Related papers  (2026-05-15T04:31:06Z)
• Mini-BEHAVIOR-Gran: Revealing U-Shaped Effects of Instruction Granularity on Language-Guided Embodied Agents [58.64789157580261]
  Mini-BEHAVIOR-Gran is a new benchmark for controlled studies of instruction granularity in embodied AI.<n>We compare four candidate metrics for cross-task granularity quantification.<n>Using width to organize training and evaluation reveals a non-monotonic U-shaped relationship between instruction granularity and performance.
  arXiv  Detail & Related papers  (2026-04-18T14:57:51Z)
• Learn to Rank: Visual Attribution by Learning Importance Ranking [58.69028273772474]
  We propose a new approach to generate visual attribution maps for computer vision models.<n>Our method produces dense, pixel-level attributions in a single forward pass with optional, few-step gradient refinement.<n>Our experiments demonstrate consistent quantitative improvements and sharper, boundary-aligned explanations.
  arXiv  Detail & Related papers  (2026-04-07T12:53:22Z)
• DISCO: Disentangled Communication Steering for Large Language Models [3.4065590965511436]
  We propose to inject steering vectors directly into the query and value representation spaces within attention heads.<n>We analytically characterize the effect of our method, which we term DISentangled COmmunication (DISCO) Steering, on attention head outputs.
  arXiv  Detail & Related papers  (2025-09-20T21:56:03Z)
• CKAA: Cross-subspace Knowledge Alignment and Aggregation for Robust Continual Learning [80.18781219542016]
  Continual Learning (CL) empowers AI models to continuously learn from sequential task streams.<n>Recent parameter-efficient fine-tuning (PEFT)-based CL methods have garnered increasing attention due to their superior performance.<n>We propose Cross-subspace Knowledge Alignment and Aggregation (CKAA) to enhance robustness against misleading task-ids.
  arXiv  Detail & Related papers  (2025-07-13T03:11:35Z)
• REAL: Reading Out Transformer Activations for Precise Localization in Language Model Steering [26.428347164111926]
  Inference-time steering aims to alter a large language model's responses without changing its parameters.<n>Existing approaches often rely on simplistic cues or ad hoc generalizations.<n>We introduce REAL, a framework for identifying behavior-relevant modules in Transformer models.
  arXiv  Detail & Related papers  (2025-06-10T02:16:50Z)
• Comparing Bottom-Up and Top-Down Steering Approaches on In-Context Learning Tasks [7.363811256593798]
  Key objective of interpretability research on large language models is to develop methods for robustly steering models toward desired behaviors. We present a case study comparing the effectiveness of representative vector steering methods from each branch. We find that ICVs outperform FVs in behavioral shifting, whereas FVs excel in tasks requiring more precision.
  arXiv  Detail & Related papers  (2024-11-11T18:36:17Z)
• Knowledge Composition using Task Vectors with Learned Anisotropic Scaling [51.4661186662329]
  We introduce aTLAS, an algorithm that linearly combines parameter blocks with different learned coefficients, resulting in anisotropic scaling at the task vector level. We show that such linear combinations explicitly exploit the low intrinsicity of pre-trained models, with only a few coefficients being the learnable parameters. We demonstrate the effectiveness of our method in task arithmetic, few-shot recognition and test-time adaptation, with supervised or unsupervised objectives.
  arXiv  Detail & Related papers  (2024-07-03T07:54:08Z)
• LoFiT: Localized Fine-tuning on LLM Representations [60.99814930367597]
  We introduce a framework called Localized Fine-Tuning on LLM Representations (LoFiT) LoFiT identifies a subset of attention heads that are most important for learning a specific task, then trains offset vectors to add to the model's hidden representations at those selected heads. For truthfulness and reasoning tasks, we find that LoFiT's intervention vectors are more effective for LLM adaptation than vectors from representation intervention methods such as Inference-time Intervention.
  arXiv  Detail & Related papers  (2024-06-03T17:45:41Z)
• Beyond Prototypes: Semantic Anchor Regularization for Better Representation Learning [82.29761875805369]
  One of the ultimate goals of representation learning is to achieve compactness within a class and well-separability between classes. We propose a novel perspective to use pre-defined class anchors serving as feature centroid to unidirectionally guide feature learning. The proposed Semantic Anchor Regularization (SAR) can be used in a plug-and-play manner in the existing models.
  arXiv  Detail & Related papers  (2023-12-19T05:52:38Z)
• Scalable Rule-Based Representation Learning for Interpretable Classification [12.736847587988853]
  Rule-based Learner Representation (RRL) learns interpretable non-fuzzy rules for data representation and classification. RRL can be easily adjusted to obtain a trade-off between classification accuracy and model complexity for different scenarios.
  arXiv  Detail & Related papers  (2021-09-30T13:07:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.