Persona Vectors: Monitoring and Controlling Character Traits in Language Models

• URL: http://arxiv.org/abs/2507.21509v3
• Date: Fri, 05 Sep 2025 07:44:31 GMT
• Title: Persona Vectors: Monitoring and Controlling Character Traits in Language Models
• Abstract summary: Large language models interact with users through a simulated 'Assistant' persona.<n>We identify directions in the model's activation space-persona vectors.<n>We apply persona vectors to predict and control personality shifts that occur during training.
• Score: 11.039979968884575
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models interact with users through a simulated 'Assistant' persona. While the Assistant is typically trained to be helpful, harmless, and honest, it sometimes deviates from these ideals. In this paper, we identify directions in the model's activation space-persona vectors-underlying several traits, such as evil, sycophancy, and propensity to hallucinate. We confirm that these vectors can be used to monitor fluctuations in the Assistant's personality at deployment time. We then apply persona vectors to predict and control personality shifts that occur during training. We find that both intended and unintended personality changes after finetuning are strongly correlated with shifts along the relevant persona vectors. These shifts can be mitigated through post-hoc intervention, or avoided in the first place with a new preventative steering method. Moreover, persona vectors can be used to flag training data that will produce undesirable personality changes, both at the dataset level and the individual sample level. Our method for extracting persona vectors is automated and can be applied to any personality trait of interest, given only a natural-language description.

Related papers

• Causal Interventions on Continuous Variables: A Case Study on Verb Bias in Steering Vectors for In-Context Learning [7.320379735876713]
  We introduce a method for causal intervention on continuous variables.<n>We apply this method to a continuous feature that is well-studied in psycholinguistics, namely verb bias.<n>We show that verb bias is causally represented in steering vectors extracted from large language models.
  arXiv  Detail & Related papers  (2026-05-28T14:09:18Z)
• The Illusion of Intervention: Your LLM-Simulated Experiment is an Observational Study [56.649987197786096]
  Large language models (LLMs) show potential as simulators of human behavior.<n>LLMs can induce unintended shifts in latent user attributes, causing user drift.<n>We formalize the confounding or selection bias that can arise due to user drift.
  arXiv  Detail & Related papers  (2026-05-20T06:09:41Z)
• Tracing Persona Vectors Through LLM Pretraining [16.7064056526422]
  How large language models internally represent high-level behaviors is a core interpretability question with direct relevance to AI safety.<n>Recent work has shown that traits such as evil or sycophancy correspond to linear directions in the internal activations, the so-called persona vectors.<n>We find that persona vectors form remarkably early -- within 0.22% of OLMo-3 pretraining -- and remain effective for steering the fully post-trained instruct models.
  arXiv  Detail & Related papers  (2026-05-13T10:44:23Z)
• Do Linear Probes Generalize Better in Persona Coordinates? [0.0]
  We study whether there exists a low-dimensional subspace of the model internals that captures harmful behaviors more robustly.<n>We construct persona axes for deception and sycophancy using contrastive persona prompts.<n>We show that persona-derived directions transfer non-trivially and probes trained on persona-PC projections generalize better than probes trained on raw activations.
  arXiv  Detail & Related papers  (2026-05-10T07:38:34Z)
• PERSONA: Dynamic and Compositional Inference-Time Personality Control via Activation Vector Algebra [84.59328460968872]
  Current methods for personality control in Large Language Models rely on static prompting or expensive fine-tuning.<n>We introduce PERSONA, a training-free framework that achieves fine-tuning level performance through direct manipulation of personality vectors.<n>On PersonalityBench, our approach achieves a mean score of 9.60, nearly matching the supervised fine-tuning upper bound of 9.61 without any gradient updates.
  arXiv  Detail & Related papers  (2026-02-17T15:47:58Z)
• The Assistant Axis: Situating and Stabilizing the Default Persona of Language Models [1.5247897700689181]
  We investigate the structure of the space of model personas by extracting activation directions corresponding to diverse character archetypes.<n>We find that the leading component of this persona space is an "Assistant Axis," which captures the extent to which a model is operating in its default Assistant mode.<n>Measuring deviations along the Assistant Axis predicts "persona drift," a phenomenon where models slip into exhibiting harmful or bizarre behaviors uncharacteristic of their typical persona.
  arXiv  Detail & Related papers  (2026-01-15T13:40:06Z)
• PersonaDrift: A Benchmark for Temporal Anomaly Detection in Language-Based Dementia Monitoring [0.9668407688201359]
  PersonaDrift is a benchmark designed to evaluate machine learning and statistical methods for detecting progressive changes in daily communication.<n>The benchmark focuses on two forms of longitudinal change that caregivers highlighted as particularly salient: flattened sentiment and off-topic replies.<n>Preliminary results show that flattened sentiment can often be detected with simple statistical models in users with low baseline variability.
  arXiv  Detail & Related papers  (2025-11-20T15:15:00Z)
• Impatient Users Confuse AI Agents: High-fidelity Simulations of Human Traits for Testing Agents [58.00130492861884]
  TraitBasis is a lightweight, model-agnostic method for systematically stress testing AI agents.<n>TraitBasis learns directions in activation space corresponding to steerable user traits.<n>We observe on average a 2%-30% performance degradation on $tau$-Trait across frontier models.
  arXiv  Detail & Related papers  (2025-10-06T05:03:57Z)
• Personality Vector: Modulating Personality of Large Language Models by Model Merging [2.375715682799016]
  We propose a novel method for personality modulation in large language models (LLMs)<n>We construct personality vectors by subtracting the weights of a pre-trained model from those of the fine-tuned model on a given personality trait.<n>Experiments show that personality vectors enable continuous control over trait intensity and support the composition of multiple traits.
  arXiv  Detail & Related papers  (2025-09-24T03:11:28Z)
• Activation Scaling for Steering and Interpreting Language Models [55.59689963561315]
  We argue that successfully intervening on a model is a prerequisite for interpreting its internal workings. We establish a three-term objective: a successful intervention should flip the correct with the wrong token and vice versa. Using gradient-based optimization, this objective lets us learn (and later evaluate) a specific kind of efficient and interpretable intervention.
  arXiv  Detail & Related papers  (2024-10-07T12:01:32Z)
• Steering Without Side Effects: Improving Post-Deployment Control of Language Models [61.99293520621248]
  Language models (LMs) have been shown to behave unexpectedly post-deployment. We present KL-then-steer (KTS), a technique that decreases the side effects of steering while retaining its benefits. Our best method prevents 44% of jailbreak attacks compared to the original Llama-2-chat-7B model.
  arXiv  Detail & Related papers  (2024-06-21T01:37:39Z)
• Words in Motion: Extracting Interpretable Control Vectors for Motion Transformers [1.1510009152620668]
  Transformer-based models generate hidden states that are difficult to interpret.<n>We analyze hidden states and modify them at inference, with a focus on motion forecasting.
  arXiv  Detail & Related papers  (2024-06-17T15:07:55Z)
• Social-Transmotion: Promptable Human Trajectory Prediction [65.80068316170613]
  Social-Transmotion is a generic Transformer-based model that exploits diverse and numerous visual cues to predict human behavior.<n>Our approach is validated on multiple datasets, including JTA, JRDB, Pedestrians and Cyclists in Road Traffic, and ETH-UCY.
  arXiv  Detail & Related papers  (2023-12-26T18:56:49Z)
• Backpack Language Models [108.65930795825416]
  We present Backpacks, a new neural architecture that marries strong modeling performance with an interface for interpretability and control. We find that, after training, sense vectors specialize, each encoding a different aspect of a word. We present simple algorithms that intervene on sense vectors to perform controllable text generation and debiasing.
  arXiv  Detail & Related papers  (2023-05-26T09:26:23Z)
• Strategic Classification under Unknown Personalized Manipulation [32.710715217516025]
  We study the fundamental mistake bound and sample complexity in the strategic classification. Agents can strategically manipulate their feature vector up to an extent in order to be predicted as positive.
  arXiv  Detail & Related papers  (2023-05-25T22:07:29Z)
• Learning Bidirectional Action-Language Translation with Limited Supervision and Incongruent Extra Input [14.548576165754804]
  We model a weakly supervised learning paradigm using our Paired Gated Autoencoders (PGAE) model. We introduce the Paired Transformed Autoencoders (PTAE) model, using Transformer-based crossmodal attention. PTAE achieves significantly higher accuracy in language-to-action and action-to-language translations.
  arXiv  Detail & Related papers  (2023-01-09T14:09:09Z)
• A unified framework for dataset shift diagnostics [2.449909275410288]
  Supervised learning techniques typically assume training data originates from the target population. Yet, dataset shift frequently arises, which, if not adequately taken into account, may decrease the performance of their predictors. We propose a novel and flexible framework called DetectShift that quantifies and tests for multiple dataset shifts.
  arXiv  Detail & Related papers  (2022-05-17T13:34:45Z)
• Extracting Latent Steering Vectors from Pretrained Language Models [14.77762401765532]
  We show that latent vectors can be extracted directly from language model decoders without fine-tuning. Experiments show that there exist steering vectors, which, when added to the hidden states of the language model, generate a target sentence nearly perfectly. We find that distances between steering vectors reflect sentence similarity when evaluated on a textual similarity benchmark.
  arXiv  Detail & Related papers  (2022-05-10T19:04:37Z)
• Learning Theory of Mind via Dynamic Traits Attribution [59.9781556714202]
  We propose a new neural ToM architecture that learns to generate a latent trait vector of an actor from the past trajectories. This trait vector then multiplicatively modulates the prediction mechanism via a fast weights' scheme in the prediction neural network. We empirically show that the fast weights provide a good inductive bias to model the character traits of agents and hence improves mindreading ability.
  arXiv  Detail & Related papers  (2022-04-17T11:21:18Z)
• Towards Building a Personalized Dialogue Generator via Implicit User Persona Detection [0.0]
  We consider high-quality transmission is essentially built based on apprehending the persona of the other party. Motivated by this, we propose a novel personalized dialogue generator by detecting implicit user persona.
  arXiv  Detail & Related papers  (2022-04-15T08:12:10Z)
• Learning Language and Multimodal Privacy-Preserving Markers of Mood from Mobile Data [74.60507696087966]
  Mental health conditions remain underdiagnosed even in countries with common access to advanced medical care. One promising data source to help monitor human behavior is daily smartphone usage. We study behavioral markers of daily mood using a recent dataset of mobile behaviors from adolescent populations at high risk of suicidal behaviors.
  arXiv  Detail & Related papers  (2021-06-24T17:46:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.