Stable Behavior, Limited Variation: Persona Validity in LLM Agents for Urban Sentiment Perception

• URL: http://arxiv.org/abs/2604.28048v1
• Date: Thu, 30 Apr 2026 15:59:11 GMT
• Title: Stable Behavior, Limited Variation: Persona Validity in LLM Agents for Urban Sentiment Perception
• Abstract summary: We use factorial set of personas spanning gender, economic status, political orientation, and personality to evaluate urban scene images.<n>Results show strong convergence among agents sharing a persona, indicating stable and reproducible behavior.<n>Agents also exhibit an extremity bias, collapsing intermediate sentiment categories common in human annotations.
• Score: 4.732324753420076
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large Language Models (LLMs) are increasingly used as proxies for human perception in urban analysis, yet it remains unclear whether persona prompting produces meaningful and reproducible behavioral diversity. We investigate whether distinct personas influence urban sentiment judgments generated by multimodal LLMs. Using a factorial set of personas spanning gender, economic status, political orientation, and personality, we instantiate multiple agents per persona to evaluate urban scene images from the PerceptSent dataset and assess both within-persona consistency and cross-persona variation. Results show strong convergence among agents sharing a persona, indicating stable and reproducible behavior. However, cross-persona differentiation is limited: economic status and personality induce statistically detectable but practically modest variation, while gender shows no measurable effect and political orientation only negligible impact. Agents also exhibit an extremity bias, collapsing intermediate sentiment categories common in human annotations. As a result, performance remains strong on coarse-grained polarity tasks but degrades as sentiment resolution increases, suggesting that simple label-based persona prompting does not capture fine-grained perceptual judgments. To isolate the contribution of persona conditioning, we additionally evaluate the same model without personas. Surprisingly, the no-persona model sometimes matches or exceeds persona-conditioned agreement with human labels across all task variants, suggesting that simple label-based persona prompting may add limited annotation value in this setting.

Related papers

• Rethinking the Evaluation and Optimization of LLM-Based Social Simulation [56.18577997572273]
  We use the subjectivity coefficient to analyze how accuracy-based evaluation and hard-label training fail as subjectivity grows.<n>We construct SUBJSIM, a benchmark of 19,300 contexts covering 193 annotators and 100 subjective questions.<n>Results on SUBJSIM demonstrate the advantages of our method.
  arXiv  Detail & Related papers  (2026-08-20T06:26:03Z)
• Do AI Personas Grow? Analyzing and Benchmarking Personality Evolution in LLM Agents After Life Events [34.88005801095875]
  Personality-conditioned LLM agents (PC-Agents) are increasingly used in emotional support, social simulation, and role-playing.<n>We study event-induced personality change after 11 major life events using the Big Five traits as a psychometric anchor.<n>Our results suggest that current PC-Agents simulate the mean of human personality dynamics, but not its shape.
  arXiv  Detail & Related papers  (2026-08-06T18:25:58Z)
• From Self to Other: Evaluating Demographic Perspective-Taking in LLM Hate Speech Annotation [5.762370982168011]
  Persona-conditioned Large Language Models (models prompted to adopt a specific demographic identity) have been proposed as a way to simulate diverse perspectives at scale.<n>We evaluate three aspects of human social judgement: (i) whether personas from different groups disagree in human-like ways (inter-group disagreement), (ii) whether they become more sensitive when content targets their own identity (in-group sensitivity), and (iii) whether they can accurately predict how another group would react (vicarious prediction).<n>Our results show that no model consistently captures all three dimensions, and performance is highly model-dependent and does not emerge reliably from
  arXiv  Detail & Related papers  (2026-06-04T15:09:58Z)
• The Chameleon's Limit: Investigating Persona Collapse and Homogenization in Large Language Models [12.654067061944074]
  Applications based on large language models (LLMs) require population diversity among agents.<n>We identify a pervasive failure mode we term emphPersona Collapse: agents each assigned a distinct profile nonetheless converge into a narrow behavioral mode, producing a homogeneous simulated population.
  arXiv  Detail & Related papers  (2026-04-27T17:01:48Z)
• Persona Prompting as a Lens on LLM Social Reasoning [5.001433675691563]
  For socially sensitive tasks like hate speech detection, the quality of explanations from Large Language Models (LLMs) is crucial.<n>While Persona prompting (PP) is increasingly used as a way to steer model towards user-specific generation, its effect on model rationales remains underexplored.
  arXiv  Detail & Related papers  (2026-01-28T16:41:17Z)
• The Personality Illusion: Revealing Dissociation Between Self-Reports & Behavior in LLMs [60.15472325639723]
  Personality traits have long been studied as predictors of human behavior.<n>Recent advances in Large Language Models (LLMs) suggest similar patterns may emerge in artificial systems.
  arXiv  Detail & Related papers  (2025-09-03T21:27:10Z)
• Evaluating the Simulation of Human Personality-Driven Susceptibility to Misinformation with LLMs [0.18416014644193066]
  Large language models (LLMs) make it possible to generate synthetic behavioural data at scale.<n>We evaluate the capacity of LLM agents, conditioned on Big-Five profiles, to reproduce personality-based variation in susceptibility to misinformation.
  arXiv  Detail & Related papers  (2025-06-30T08:16:07Z)
• Spotting Out-of-Character Behavior: Atomic-Level Evaluation of Persona Fidelity in Open-Ended Generation [16.76995815742803]
  We propose an atomic-level evaluation framework that quantifies persona fidelity at a finer granularity.<n>Our three key metrics measure the degree of persona alignment and consistency within and across generations.<n>By analyzing persona fidelity across diverse tasks and personality types, we reveal how task structure and persona desirability influence model adaptability.
  arXiv  Detail & Related papers  (2025-06-24T06:33:10Z)
• PAARS: Persona Aligned Agentic Retail Shoppers [2.8737584376365355]
  In e-commerce, behavioral data is collected for decision making which can be costly and slow.<n>We propose a framework that creates synthetic shopping agents by automatically mining anonymised historical shopping data.<n>We showcase an initial application of our framework for automated agentic A/B testing and compare the findings to human results.
  arXiv  Detail & Related papers  (2025-03-31T15:41:51Z)
• Beyond correlation: The Impact of Human Uncertainty in Measuring the Effectiveness of Automatic Evaluation and LLM-as-a-Judge [51.93909886542317]
  We show how *relying on a single aggregate correlation score* can obscure fundamental differences between human labels and those from automatic evaluation.<n>We propose stratifying data by human label uncertainty to provide a more robust analysis of automatic evaluation performance.
  arXiv  Detail & Related papers  (2024-10-03T03:08:29Z)
• Scaling Data Diversity for Fine-Tuning Language Models in Human Alignment [84.32768080422349]
  Alignment with human preference prevents large language models from generating misleading or toxic content. We propose a new formulation of prompt diversity, implying a linear correlation with the final performance of LLMs after fine-tuning.
  arXiv  Detail & Related papers  (2024-03-17T07:08:55Z)
• Decoding Susceptibility: Modeling Misbelief to Misinformation Through a Computational Approach [61.04606493712002]
  Susceptibility to misinformation describes the degree of belief in unverifiable claims that is not observable. Existing susceptibility studies heavily rely on self-reported beliefs. We propose a computational approach to model users' latent susceptibility levels.
  arXiv  Detail & Related papers  (2023-11-16T07:22:56Z)
• On the steerability of large language models toward data-driven personas [98.9138902560793]
  Large language models (LLMs) are known to generate biased responses where the opinions of certain groups and populations are underrepresented. Here, we present a novel approach to achieve controllable generation of specific viewpoints using LLMs.
  arXiv  Detail & Related papers  (2023-11-08T19:01:13Z)
• Sensitivity, Performance, Robustness: Deconstructing the Effect of Sociodemographic Prompting [64.80538055623842]
  sociodemographic prompting is a technique that steers the output of prompt-based models towards answers that humans with specific sociodemographic profiles would give. We show that sociodemographic information affects model predictions and can be beneficial for improving zero-shot learning in subjective NLP tasks.
  arXiv  Detail & Related papers  (2023-09-13T15:42:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.