Tracing Persona Vectors Through LLM Pretraining

• URL: http://arxiv.org/abs/2605.13329v1
• Date: Wed, 13 May 2026 10:44:23 GMT
• Title: Tracing Persona Vectors Through LLM Pretraining
• Abstract summary: How large language models internally represent high-level behaviors is a core interpretability question with direct relevance to AI safety.<n>Recent work has shown that traits such as evil or sycophancy correspond to linear directions in the internal activations, the so-called persona vectors.<n>We find that persona vectors form remarkably early -- within 0.22% of OLMo-3 pretraining -- and remain effective for steering the fully post-trained instruct models.
• Score: 16.7064056526422
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: How large language models internally represent high-level behaviors is a core interpretability question with direct relevance to AI safety: it determines what we can detect, audit, or intervene on. Recent work has shown that traits such as evil or sycophancy correspond to linear directions in the internal activations, the so-called persona vectors. Although these vectors are now routinely utilized to inspect and steer model behavior in safety-relevant settings, how these representations are formed during training remains unknown. To address this gap, we trace persona vectors across the pretraining of OLMo-3-7B, finding that persona vectors form remarkably early -- within 0.22% of OLMo-3 pretraining -- and remain effective for steering the fully post-trained instruct models. Although core representations are formed early on, persona vectors continue to refine geometrically and semantically throughout pretraining. We further compare alternative elicitation strategies and find that all yield effective directions, with each strategy surfacing qualitatively distinct facets of the underlying persona. Replicating our analysis on Apertus-8B reveals that our findings transfer qualitatively beyond OLMo-3. Our results establish persona representations as stable features of early pretraining and open a path to studying how training forms, refines, and shapes them.

Related papers

• Evaluation Awareness in Language Models: Representation, Verbalization, and Control [5.471648649900293]
  We show that evaluation awareness is already present within base models, becomes amplified throughout the stages of supervised fine-tuning, and remains stable thereafter.<n>These results show the need for evaluations to account for the disjunction between what models represent internally, what they represent, and their steering.
  arXiv  Detail & Related papers  (2026-08-22T04:16:22Z)
• PERSONA: Dynamic and Compositional Inference-Time Personality Control via Activation Vector Algebra [84.59328460968872]
  Current methods for personality control in Large Language Models rely on static prompting or expensive fine-tuning.<n>We introduce PERSONA, a training-free framework that achieves fine-tuning level performance through direct manipulation of personality vectors.<n>On PersonalityBench, our approach achieves a mean score of 9.60, nearly matching the supervised fine-tuning upper bound of 9.61 without any gradient updates.
  arXiv  Detail & Related papers  (2026-02-17T15:47:58Z)
• Open Character Training: Shaping the Persona of AI Assistants through Constitutional AI [4.153803842911732]
  "AI assistant" persona influences both surface-level behavior and apparent values, beliefs, and ethics.<n>Character training is a critical component of industry post-training, yet remains effectively unstudied in the academic literature.<n>We introduce the first open implementation of character training, leveraging Constitutional AI and a new data pipeline.
  arXiv  Detail & Related papers  (2025-11-03T15:53:47Z)
• Small Vectors, Big Effects: A Mechanistic Study of RL-Induced Reasoning via Steering Vectors [12.331740215947677]
  We study lightweight steering vectors inserted into the base model's residual stream and trained with a reinforcement-learning objective.<n>We find that (i) the last-layer steering vector acts like a token-substitution bias concentrated on the first generated token, consistently boosting tokens such as "To" and "Step"<n>We also show that steering vectors (i) transfer to other models, (ii) combine across layers when trained in isolation, and (iii) concentrate magnitude on meaningful prompt segments under adaptive token-wise scaling.
  arXiv  Detail & Related papers  (2025-09-08T12:26:31Z)
• The Personality Illusion: Revealing Dissociation Between Self-Reports & Behavior in LLMs [60.15472325639723]
  Personality traits have long been studied as predictors of human behavior.<n>Recent advances in Large Language Models (LLMs) suggest similar patterns may emerge in artificial systems.
  arXiv  Detail & Related papers  (2025-09-03T21:27:10Z)
• Prior Learning in Introspective VAEs [26.51505732100697]
  Variational Autoencoders (VAEs) are a popular framework for unsupervised learning and data generation.<n>In this study, we focus on the Soft-IntroVAE (S-IntroVAE), one of only two members of the Introspective VAE family.<n>We investigate the implication of incorporating a multimodal and trainable prior into this S-IntroVAE.
  arXiv  Detail & Related papers  (2024-08-25T10:54:25Z)
• Initialization Matters for Adversarial Transfer Learning [61.89451332757625]
  We discover the necessity of an adversarially robust pretrained model. We propose Robust Linear Initialization (RoLI) for adversarial finetuning, which initializes the linear head with the weights obtained by adversarial linear probing. Across five different image classification datasets, we demonstrate the effectiveness of RoLI and achieve new state-of-the-art results.
  arXiv  Detail & Related papers  (2023-12-10T00:51:05Z)
• A Study of Forward-Forward Algorithm for Self-Supervised Learning [65.268245109828]
  We study the performance of forward-forward vs. backpropagation for self-supervised representation learning. Our main finding is that while the forward-forward algorithm performs comparably to backpropagation during (self-supervised) training, the transfer performance is significantly lagging behind in all the studied settings.
  arXiv  Detail & Related papers  (2023-09-21T10:14:53Z)
• SignBERT+: Hand-model-aware Self-supervised Pre-training for Sign Language Understanding [132.78015553111234]
  Hand gesture serves as a crucial role during the expression of sign language. Current deep learning based methods for sign language understanding (SLU) are prone to over-fitting due to insufficient sign data resource. We propose the first self-supervised pre-trainable SignBERT+ framework with model-aware hand prior incorporated.
  arXiv  Detail & Related papers  (2023-05-08T17:16:38Z)
• On the Transferability of Pre-trained Language Models: A Study from Artificial Datasets [74.11825654535895]
  Pre-training language models (LMs) on large-scale unlabeled text data makes the model much easier to achieve exceptional downstream performance. We study what specific traits in the pre-training data, other than the semantics, make a pre-trained LM superior to their counterparts trained from scratch on downstream tasks.
  arXiv  Detail & Related papers  (2021-09-08T10:39:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.