When Persona Attributes Improve Population Alignment in Large Language Models

• URL: http://arxiv.org/abs/2609.02526v1
• Date: Wed, 02 Sep 2026 12:30:56 GMT
• Title: When Persona Attributes Improve Population Alignment in Large Language Models
• Abstract summary: Persona prompting refers to the practice of using short textual descriptions of 'personas' in prompts to steer large language models.<n>Recent work has produced mixed and partly conflicting results of persona prompting without clear patterns of success and failure.<n>It remains unclear how different attribute selection methods perform and how to choose among them.
• Score: 2.416660506672577
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large Language Models (LLMs) are increasingly used to predict the responses of human participants in survey panels. Towards that goal, persona prompting has recently emerged as a technique to inform and align large pretrained language models. Persona prompting refers to the practice of using short textual descriptions of 'personas' in prompts to steer the LLM's generations. Personas describe individuals through different attributes such as their socio-demographics, attitudes, or behaviors, with the aim of aligning LLMs to produce responses that correlate with the corresponding human responses. Yet, recent work has produced mixed and partly conflicting results of persona prompting without clear patterns of success and failure. Among the few consistent findings is that the selection of persona attributes matters, and that using more attributes does not necessarily lead to better performance. It remains unclear how different attribute selection methods perform and how to choose among them. In this paper, we propose that observed human response variation of a survey question is a potential explanation for the mixed performance observed so far. In addition, we compare the performance of persona prompting associated with different methods for selecting persona attributes. We evaluate these methods on four different (general) social surveys across two countries, six LLMs, and twenty prediction tasks per survey. Our work helps to identify when persona prompting can be expected to be useful in survey prediction tasks, and provides new insights on the effectiveness of different attribute selection methods for LLM-based survey prediction using persona prompting.

Related papers

• From Classification to Ranking: Enhancing LLM Reasoning Capabilities for MBTI Personality Detection [25.825456002235967]
  Personality detection aims to measure an individual's corresponding personality traits through their social media posts.<n>Existing approaches enhance personality trait analysis by leveraging Large Language Models (LLMs)<n>We propose a corresponding reinforcement learning training paradigm for personality detection.
  arXiv  Detail & Related papers  (2026-01-26T15:28:43Z)
• One Persona, Many Cues, Different Results: How Sociodemographic Cues Impact LLM Personalization [6.512258839228369]
  Personalization of LLMs by sociodemographic subgroup often improves user experience, but can also introduce or amplify biases and unfair outcomes across groups.<n>Prior work has employed so-called personas, sociodemographic user attributes conveyed to a model, to study bias in LLMs.<n>We compare six commonly used persona cues across seven open and proprietary LLMs on four writing and advice tasks.<n>While cues are overall highly correlated, they produce substantial variance in responses across personas.
  arXiv  Detail & Related papers  (2026-01-26T15:15:58Z)
• What Makes a Good Natural Language Prompt? [72.3282960118995]
  We conduct a meta-analysis surveying more than 150 prompting-related papers from leading NLP and AI conferences from 2022 to 2025.<n>We propose a property- and human-centric framework for evaluating prompt quality, encompassing 21 properties categorized into six dimensions.<n>We then empirically explore multi-property prompt enhancements in reasoning tasks, observing that single-property enhancements often have the greatest impact.
  arXiv  Detail & Related papers  (2025-06-07T23:19:27Z)
• Deep Binding of Language Model Virtual Personas: a Study on Approximating Political Partisan Misperceptions [13.929713456538932]
  We propose a novel methodology for constructing virtual personas with synthetic user "backstories" generated as extended, multi-turn interview transcripts.<n>We show that virtual personas conditioned on our backstories closely replicate human response distributions and produce effect sizes that closely match those observed in the original studies of in-group/out-group biases.
  arXiv  Detail & Related papers  (2025-04-16T00:10:34Z)
• When "A Helpful Assistant" Is Not Really Helpful: Personas in System Prompts Do Not Improve Performances of Large Language Models [34.831938712535084]
  Commercial AI systems commonly define the role of the Large Language Models (LLM) in system prompts. It remains unclear how different personas affect a model's performance on objective tasks. We curate a list of 162 roles covering 6 types of interpersonal relationships and 8 domains of expertise.
  arXiv  Detail & Related papers  (2023-11-16T17:48:55Z)
• Sociodemographic Prompting is Not Yet an Effective Approach for Simulating Subjective Judgments with LLMs [13.744746481528711]
  Large Language Models (LLMs) are widely used to simulate human responses across diverse contexts.<n>We evaluate nine popular LLMs on their ability to understand demographic differences in two subjective judgment tasks: politeness and offensiveness.<n>We find that in zero-shot settings, most models' predictions for both tasks align more closely with labels from White participants than those from Asian or Black participants.
  arXiv  Detail & Related papers  (2023-11-16T10:02:24Z)
• Do LLMs exhibit human-like response biases? A case study in survey design [66.1850490474361]
  We investigate the extent to which large language models (LLMs) reflect human response biases, if at all. We design a dataset and framework to evaluate whether LLMs exhibit human-like response biases in survey questionnaires. Our comprehensive evaluation of nine models shows that popular open and commercial LLMs generally fail to reflect human-like behavior.
  arXiv  Detail & Related papers  (2023-11-07T15:40:43Z)
• PsyCoT: Psychological Questionnaire as Powerful Chain-of-Thought for Personality Detection [50.66968526809069]
  We propose a novel personality detection method, called PsyCoT, which mimics the way individuals complete psychological questionnaires in a multi-turn dialogue manner. Our experiments demonstrate that PsyCoT significantly improves the performance and robustness of GPT-3.5 in personality detection.
  arXiv  Detail & Related papers  (2023-10-31T08:23:33Z)
• Sensitivity, Performance, Robustness: Deconstructing the Effect of Sociodemographic Prompting [64.80538055623842]
  sociodemographic prompting is a technique that steers the output of prompt-based models towards answers that humans with specific sociodemographic profiles would give. We show that sociodemographic information affects model predictions and can be beneficial for improving zero-shot learning in subjective NLP tasks.
  arXiv  Detail & Related papers  (2023-09-13T15:42:06Z)
• Can ChatGPT Assess Human Personalities? A General Evaluation Framework [70.90142717649785]
  Large Language Models (LLMs) have produced impressive results in various areas, but their potential human-like psychology is still largely unexplored. This paper presents a generic evaluation framework for LLMs to assess human personalities based on Myers Briggs Type Indicator (MBTI) tests.
  arXiv  Detail & Related papers  (2023-03-01T06:16:14Z)
• Partner Matters! An Empirical Study on Fusing Personas for Personalized Response Selection in Retrieval-Based Chatbots [51.091235903442715]
  This paper makes an attempt to explore the impact of utilizing personas that describe either self or partner speakers on the task of response selection. Four persona fusion strategies are designed, which assume personas interact with contexts or responses in different ways. Empirical studies on the Persona-Chat dataset show that the partner personas can improve the accuracy of response selection.
  arXiv  Detail & Related papers  (2021-05-19T10:32:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.