Personality Matters: User Traits Predict LLM Preferences in Multi-Turn Collaborative Tasks

• URL: http://arxiv.org/abs/2508.21628v1
• Date: Fri, 29 Aug 2025 13:42:26 GMT
• Title: Personality Matters: User Traits Predict LLM Preferences in Multi-Turn Collaborative Tasks
• Authors: Sarfaroz Yunusov, Kaige Chen, Kazi Nishat Anwar, Ali Emami,
• Abstract summary: Large Language Models (LLMs) increasingly integrate into everyday, where users shape outcomes through multi-turn collaboration.<n>Do users with different personality traits systematically prefer certain LLMs over others?<n>We conducted a study with 32 participants evenly distributed across four Keirsey personality types, evaluating interactions with GPT-4 and Claude 3.5.
• Score: 11.841394824977984
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As Large Language Models (LLMs) increasingly integrate into everyday workflows, where users shape outcomes through multi-turn collaboration, a critical question emerges: do users with different personality traits systematically prefer certain LLMs over others? We conducted a study with 32 participants evenly distributed across four Keirsey personality types, evaluating their interactions with GPT-4 and Claude 3.5 across four collaborative tasks: data analysis, creative writing, information retrieval, and writing assistance. Results revealed significant personality-driven preferences: Rationals strongly preferred GPT-4, particularly for goal-oriented tasks, while idealists favored Claude 3.5, especially for creative and analytical tasks. Other personality types showed task-dependent preferences. Sentiment analysis of qualitative feedback confirmed these patterns. Notably, aggregate helpfulness ratings were similar across models, showing how personality-based analysis reveals LLM differences that traditional evaluations miss.

Related papers

• Towards Realistic Personalization: Evaluating Long-Horizon Preference Following in Personalized User-LLM Interactions [50.70965714314064]
  Large Language Models (LLMs) are increasingly serving as personal assistants, where users share complex and diverse preferences over extended interactions.<n>This work proposes RealPref, a benchmark for evaluating realistic preference-following in personalized user-LLM interactions.
  arXiv  Detail & Related papers  (2026-03-04T15:42:43Z)
• Personalities at Play: Probing Alignment in AI Teammates [1.0742675209112622]
  Large language models (LLMs) increasingly function as collaborators rather than tools.<n>We investigate AI personality alignment through a three-lens evaluation framework.<n>Results suggest that AI personality is measurable but multi-layered and context-dependent.
  arXiv  Detail & Related papers  (2026-02-28T03:06:02Z)
• Vibe Check: Understanding the Effects of LLM-Based Conversational Agents' Personality and Alignment on User Perceptions in Goal-Oriented Tasks [2.1117030125341385]
  Large language models (LLMs) enable conversational agents (CAs) to express distinctive personalities.<n>This study investigates how personality expression levels and user-agent personality alignment influence perceptions in goal-oriented tasks.
  arXiv  Detail & Related papers  (2025-09-11T21:43:49Z)
• Are Economists Always More Introverted? Analyzing Consistency in Persona-Assigned LLMs [12.780044838203738]
  We introduce a new standardized framework to analyze consistency in persona-assigned Large Language Models (LLMs)<n>Our framework evaluates personas across four different categories (happiness, occupation, personality, and political stance) spanning multiple task dimensions.<n>Our findings reveal that consistency is influenced by multiple factors, including the assigned persona, stereotypes, and model design choices.
  arXiv  Detail & Related papers  (2025-06-03T09:12:23Z)
• Aligning LLMs with Individual Preferences via Interaction [51.72200436159636]
  We train large language models (LLMs) that can ''interact to align''<n>We develop a multi-turn preference dataset containing 3K+ multi-turn conversations in tree structures.<n>For evaluation, we establish the ALOE benchmark, consisting of 100 carefully selected examples and well-designed metrics to measure the customized alignment performance during conversations.
  arXiv  Detail & Related papers  (2024-10-04T17:48:29Z)
• Personality Alignment of Large Language Models [30.710131188931317]
  Personality Alignment aims to align large language models with individual user preferences.<n>This dataset includes data from over 320,000 real subjects across multiple personality assessments.<n>We develop an activation intervention optimization method to efficiently align with individual behavioral preferences.<n>Our work paves the way for future AI systems to make decisions and reason in truly personality ways.
  arXiv  Detail & Related papers  (2024-08-21T17:09:00Z)
• Evaluating Large Language Models with Psychometrics [59.821829073478376]
  This paper offers a comprehensive benchmark for quantifying psychological constructs of Large Language Models (LLMs)<n>Our work identifies five key psychological constructs -- personality, values, emotional intelligence, theory of mind, and self-efficacy -- assessed through a suite of 13 datasets.<n>We uncover significant discrepancies between LLMs' self-reported traits and their response patterns in real-world scenarios, revealing complexities in their behaviors.
  arXiv  Detail & Related papers  (2024-06-25T16:09:08Z)
• PsyCoT: Psychological Questionnaire as Powerful Chain-of-Thought for Personality Detection [50.66968526809069]
  We propose a novel personality detection method, called PsyCoT, which mimics the way individuals complete psychological questionnaires in a multi-turn dialogue manner. Our experiments demonstrate that PsyCoT significantly improves the performance and robustness of GPT-3.5 in personality detection.
  arXiv  Detail & Related papers  (2023-10-31T08:23:33Z)
• Editing Personality for Large Language Models [73.59001811199823]
  This paper introduces an innovative task focused on editing the personality traits of Large Language Models (LLMs) We construct PersonalityEdit, a new benchmark dataset to address this task.
  arXiv  Detail & Related papers  (2023-10-03T16:02:36Z)
• Enhancing Large Language Models in Coding Through Multi-Perspective Self-Consistency [127.97467912117652]
  Large language models (LLMs) have exhibited remarkable ability in code generation. However, generating the correct solution in a single attempt still remains a challenge. We propose the Multi-Perspective Self-Consistency (MPSC) framework incorporating both inter- and intra-consistency.
  arXiv  Detail & Related papers  (2023-09-29T14:23:26Z)
• PersonaLLM: Investigating the Ability of Large Language Models to Express Personality Traits [30.770525830385637]
  We study the behavior of large language models (LLMs) based on the Big Five personality model. Results show that LLM personas' self-reported BFI scores are consistent with their designated personality types. Human evaluation shows that humans can perceive some personality traits with an accuracy of up to 80%.
  arXiv  Detail & Related papers  (2023-05-04T04:58:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.