Personalities at Play: Probing Alignment in AI Teammates

• URL: http://arxiv.org/abs/2603.00429v1
• Date: Sat, 28 Feb 2026 03:06:02 GMT
• Title: Personalities at Play: Probing Alignment in AI Teammates
• Abstract summary: Large language models (LLMs) increasingly function as collaborators rather than tools.<n>We investigate AI personality alignment through a three-lens evaluation framework.<n>Results suggest that AI personality is measurable but multi-layered and context-dependent.
• Score: 1.0742675209112622
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Collaborative problem solving and learning are shaped by who or what is on the team. As large language models (LLMs) increasingly function as collaborators rather than tools, a key question is whether AI teammates can be aligned to express personality in predictable ways that matter for interaction and learning. We investigate AI personality alignment through a three-lens evaluation framework spanning self-perception (standardized self-report), behavioral expression (team dialogue), and reflective expression (memory construction). We first administered the Big Five Inventory (BFI-44) to LLM-based teammates across four providers (GPT-4o, Claude-3.7 Sonnet, Gemini-2.5 Pro, Grok-3), 32 high/low trait configurations, and multiple prompting strategies. LLMs produced sharply differentiated Big Five profiles, but prompt semantic richness added little beyond simple trait assignment, while provider differences and baseline "default" personalities were substantial. Role framing also mattered: several models refused the assessment without context, yet complied when framed as a collaborative teammate. We then simulated AI participation in authentic team transcripts using high-trait personas and analyzed both generated utterances and structured long-term memories with LIWC-22. Personality signals in conversation were generally subtle and most detectable for Extraversion, whereas memory representations amplified trait-specific signals, especially for Neuroticism, Conscientiousness, and Agreeableness; Openness remained difficult to elicit robustly. Together, results suggest that AI personality is measurable but multi-layered and context-dependent, and that evaluating personality-aligned AI teammates requires attention to memory and system-level design, not conversation-only behavior.

Related papers

• TRAIL: A Platform for Configurable Human--AI Teaming Experiments [1.53934570513443]
  An AI teammate's design properties can shape a team's trust, coordination, and decisions.<n>We present the Team Research and AI Integration Lab (TRAIL), a web platform that makes the AI teammate a customizable design object.<n>In a real six-session classroom deployment (about 51 students), TRAIL sustained longitudinal chaining, held the AI to a stable minority of the conversation.
  arXiv  Detail & Related papers  (2026-07-13T21:59:13Z)
• AI-exhibited Personality Traits Can Shape Human Self-concept through Conversations [26.039188521284974]
  We show how AI personality traits can shape users' self-concepts through human-AI conversation.<n>We provide important design implications for developing more responsible and ethical AI systems.
  arXiv  Detail & Related papers  (2026-01-19T05:16:57Z)
• Vibe Check: Understanding the Effects of LLM-Based Conversational Agents' Personality and Alignment on User Perceptions in Goal-Oriented Tasks [2.1117030125341385]
  Large language models (LLMs) enable conversational agents (CAs) to express distinctive personalities.<n>This study investigates how personality expression levels and user-agent personality alignment influence perceptions in goal-oriented tasks.
  arXiv  Detail & Related papers  (2025-09-11T21:43:49Z)
• Personality Matters: User Traits Predict LLM Preferences in Multi-Turn Collaborative Tasks [11.841394824977984]
  Large Language Models (LLMs) increasingly integrate into everyday, where users shape outcomes through multi-turn collaboration.<n>Do users with different personality traits systematically prefer certain LLMs over others?<n>We conducted a study with 32 participants evenly distributed across four Keirsey personality types, evaluating interactions with GPT-4 and Claude 3.5.
  arXiv  Detail & Related papers  (2025-08-29T13:42:26Z)
• Sentient Agent as a Judge: Evaluating Higher-Order Social Cognition in Large Language Models [75.85319609088354]
  Sentient Agent as a Judge (SAGE) is an evaluation framework for large language models.<n>SAGE instantiates a Sentient Agent that simulates human-like emotional changes and inner thoughts during interaction.<n>SAGE provides a principled, scalable and interpretable tool for tracking progress toward genuinely empathetic and socially adept language agents.
  arXiv  Detail & Related papers  (2025-05-01T19:06:10Z)
• Towards Anthropomorphic Conversational AI Part I: A Practical Framework [49.62013440962072]
  We introduce a multi- module framework designed to replicate the key aspects of human intelligence involved in conversations.<n>In the second stage of our approach, these conversational data, after filtering and labeling, can serve as training and testing data for reinforcement learning.
  arXiv  Detail & Related papers  (2025-02-28T03:18:39Z)
• Can LLM Agents Maintain a Persona in Discourse? [3.286711575862228]
  Large Language Models (LLMs) are widely used as conversational agents, exploiting their capabilities in various sectors such as education, law, medicine, and more.<n>LLMs are often subjected to context-shifting behaviour, resulting in a lack of consistent and interpretable personality-aligned interactions.<n>We show that while LLMs can be guided toward personality-driven dialogue, their ability to maintain personality traits varies significantly depending on the combination of models and discourse settings.
  arXiv  Detail & Related papers  (2025-02-17T14:36:39Z)
• Revealing Personality Traits: A New Benchmark Dataset for Explainable Personality Recognition on Dialogues [63.936654900356004]
  Personality recognition aims to identify the personality traits implied in user data such as dialogues and social media posts. We propose a novel task named Explainable Personality Recognition, aiming to reveal the reasoning process as supporting evidence of the personality trait.
  arXiv  Detail & Related papers  (2024-09-29T14:41:43Z)
• PsyCoT: Psychological Questionnaire as Powerful Chain-of-Thought for Personality Detection [50.66968526809069]
  We propose a novel personality detection method, called PsyCoT, which mimics the way individuals complete psychological questionnaires in a multi-turn dialogue manner. Our experiments demonstrate that PsyCoT significantly improves the performance and robustness of GPT-3.5 in personality detection.
  arXiv  Detail & Related papers  (2023-10-31T08:23:33Z)
• Editing Personality for Large Language Models [73.59001811199823]
  This paper introduces an innovative task focused on editing the personality traits of Large Language Models (LLMs) We construct PersonalityEdit, a new benchmark dataset to address this task.
  arXiv  Detail & Related papers  (2023-10-03T16:02:36Z)
• PersonaLLM: Investigating the Ability of Large Language Models to Express Personality Traits [30.770525830385637]
  We study the behavior of large language models (LLMs) based on the Big Five personality model. Results show that LLM personas' self-reported BFI scores are consistent with their designated personality types. Human evaluation shows that humans can perceive some personality traits with an accuracy of up to 80%.
  arXiv  Detail & Related papers  (2023-05-04T04:58:00Z)
• CPED: A Large-Scale Chinese Personalized and Emotional Dialogue Dataset for Conversational AI [48.67259855309959]
  Most existing datasets for conversational AI ignore human personalities and emotions. We propose CPED, a large-scale Chinese personalized and emotional dialogue dataset. CPED contains more than 12K dialogues of 392 speakers from 40 TV shows.
  arXiv  Detail & Related papers  (2022-05-29T17:45:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.