The Impact of Steering Large Language Models with Persona Vectors in Educational Applications

• URL: http://arxiv.org/abs/2604.07102v1
• Date: Wed, 08 Apr 2026 13:55:32 GMT
• Title: The Impact of Steering Large Language Models with Persona Vectors in Educational Applications
• Abstract summary: Activation-based steering can personalize large language models at inference time.<n>This study systematically examines the effects of activation-steered persona traits in educational generation and scoring.
• Score: 2.2917707112773598
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Activation-based steering can personalize large language models at inference time, but its effects in educational settings remain unclear. We study persona vectors for seven character traits in short-answer generation and automated scoring on the ASAP-SAS benchmark across three models spanning two architectures. Persona steering lowers answer quality overall, with much larger effects on open-ended English Language Arts (ELA) prompts than on factual science prompts; interpretive and argumentative tasks are up to 11x more sensitive. On the scoring side, we observe predictable valence-aligned calibration shifts: evil and impolite scorers grade more harshly, while good and optimistic scorers grade more leniently. ELA tasks are 2.5-3x more susceptible to scorer personalization than science tasks, and the Mixture-of-Experts model shows roughly 6x larger calibration shifts than the dense models. To our knowledge, this is the first study to systematically examine the effects of activation-steered persona traits in educational generation and scoring, and the results highlight the need for task-aware and architecture-aware calibration when deploying steered models in educational settings.

Related papers

• Investigating Linguistic Steering: An Analysis of Adjectival Effects Across Large Language Model Architectures [0.0]
  We use Shapley values to quantify the steering effect of individual adjectives on model performance.<n>We find that a small subset of adjectives act as disproportionately powerful "levers," yet their effects are not universal.<n>Cross-model analysis reveals a "family effect," while architecturally distinct models react in a largely uncorrelated manner.
  arXiv  Detail & Related papers  (2026-04-28T12:14:30Z)
• LIBERO-Plus: In-depth Robustness Analysis of Vision-Language-Action Models [49.92148175114169]
  We perform a systematic vulnerability analysis by introducing controlled perturbations across seven dimensions.<n>Models exhibit extreme sensitivity to perturbation factors, including camera viewpoints and robot initial states.<n>Surprisingly, models are largely insensitive to language variations, with further experiments revealing that models tend to ignore language instructions completely.
  arXiv  Detail & Related papers  (2025-10-15T14:51:36Z)
• Steering Language Models in Multi-Token Generation: A Case Study on Tense and Aspect [3.360832612971853]
  We study the representation and control of two multidimensional hierarchical grammar phenomena - verb tense and aspect.<n>We show causal control over both grammatical features through concept steering across three generation tasks.<n>We find that steering strength, location, and duration are crucial parameters for reducing undesirable side effects such as topic shift.
  arXiv  Detail & Related papers  (2025-09-15T15:48:09Z)
• The Impact of Model Scaling on Seen and Unseen Language Performance [2.012425476229879]
  We study the performance and scaling behavior of multilingual Large Language Models across 204 languages.<n>Our findings show significant differences in scaling behavior between zero-shot and two-shot scenarios.<n>In two-shot settings, larger models show clear linear improvements in multilingual text classification.
  arXiv  Detail & Related papers  (2025-01-10T00:10:21Z)
• Harnessing the Intrinsic Knowledge of Pretrained Language Models for Challenging Text Classification Settings [5.257719744958367]
  This thesis explores three challenging settings in text classification by leveraging the intrinsic knowledge of pretrained language models (PLMs) We develop models that utilize features based on contextualized word representations from PLMs, achieving performance that rivals or surpasses human accuracy. Lastly, we tackle the sensitivity of large language models to in-context learning prompts by selecting effective demonstrations.
  arXiv  Detail & Related papers  (2024-08-28T09:07:30Z)
• Program-Aided Reasoners (better) Know What They Know [59.29201607431494]
  We compare the calibration of Program Aided Language Models (PAL) and text-based Chain-of-thought (COT) prompting techniques over 5 datasets. Our results indicate that PAL leads to improved calibration in 75% of the instances.
  arXiv  Detail & Related papers  (2023-11-16T04:17:49Z)
• Dynamics of Instruction Fine-Tuning for Chinese Large Language Models [19.832906541004114]
  We systematically investigate the effects of data quantity, model size, and data construction methods on instruction tuning for Chinese Large Language Models.<n>Our experiments, conducted on models ranging from 7b to 33b parameters, yield three key findings.
  arXiv  Detail & Related papers  (2023-10-30T15:37:10Z)
• A Comprehensive Evaluation and Analysis Study for Chinese Spelling Check [53.152011258252315]
  We show that using phonetic and graphic information reasonably is effective for Chinese Spelling Check. Models are sensitive to the error distribution of the test set, which reflects the shortcomings of models. The commonly used benchmark, SIGHAN, can not reliably evaluate models' performance.
  arXiv  Detail & Related papers  (2023-07-25T17:02:38Z)
• Large Language Models in the Workplace: A Case Study on Prompt Engineering for Job Type Classification [58.720142291102135]
  This case study investigates the task of job classification in a real-world setting. The goal is to determine whether an English-language job posting is appropriate for a graduate or entry-level position.
  arXiv  Detail & Related papers  (2023-03-13T14:09:53Z)
• Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models [648.3665819567409]
  Language models demonstrate both quantitative improvement and new qualitative capabilities with increasing scale. Big-bench consists of 204 tasks, contributed by 450 authors across 132 institutions. We evaluate the behavior of OpenAI's GPT models, Google-internal dense transformer architectures, and Switch-style sparse transformers on BIG-bench.
  arXiv  Detail & Related papers  (2022-06-09T17:05:34Z)
• Towards Trustworthy AutoGrading of Short, Multi-lingual, Multi-type Answers [2.2000998828262652]
  This study uses a large dataset consisting of about 10 million question-answer pairs from multiple languages. We show how to improve the accuracy of automatically graded answers, achieving accuracy equivalent to that of teaching assistants.
  arXiv  Detail & Related papers  (2022-01-02T12:17:24Z)
• AES Systems Are Both Overstable And Oversensitive: Explaining Why And Proposing Defenses [66.49753193098356]
  We investigate the reason behind the surprising adversarial brittleness of scoring models. Our results indicate that autoscoring models, despite getting trained as "end-to-end" models, behave like bag-of-words models. We propose detection-based protection models that can detect oversensitivity and overstability causing samples with high accuracies.
  arXiv  Detail & Related papers  (2021-09-24T03:49:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.