HEART-Bench: Do LLM Agents Exhibit Human-like Psychology?

• URL: http://arxiv.org/abs/2605.30058v1
• Date: Thu, 28 May 2026 15:08:03 GMT
• Title: HEART-Bench: Do LLM Agents Exhibit Human-like Psychology?
• Abstract summary: This paper introduces a novel benchmark to assess whether LLM agents can simulate coherent, human-like psychology.<n>Our benchmark constructs 11 diverse human characters grounded in Big Five personality traits, with each profile deeply integrated with 1,000 structured autobiographical-style episodic memories.<n>By subjecting agents to varying scenarios, the benchmark evaluates whether they can consolidate their innate personality traits and autobiographical memories to make behavioral decisions consistent with their specific psychological profiles.
• Score: 25.237337617299946
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While LLM agents have demonstrated remarkable task-oriented abilities such as planning, reasoning, and action, few works have treated them as complete human personalities where emotional dimensions hold equal importance. In this paper, we introduce a novel benchmark to systematically assess whether LLM agents can simulate coherent, human-like psychology. Specifically, our benchmark constructs 11 diverse human characters grounded in orthogonal Big Five personality traits, with each profile deeply integrated with 1,000 structured autobiographical-style episodic memories distributed across theory-grounded developmental life stages. To rigorously evaluate the psychological manifestations of LLMs, we designed a curated suite of 64 decision-making scenarios, guided by the DIAMONDS taxonomy, a psychological framework that characterizes situations along eight dimensions: Duty, Intellect, Adversity, Mating, pOsitivity, Negativity, Deception, and Sociality. By subjecting agents to varying scenarios, the benchmark evaluates whether they can consolidate their innate personality traits and autobiographical memories to make behavioral decisions that are consistent with their specific psychological profiles. After systematic human validation and filtering, we obtained a benchmark consisting of 673 multiple-choice questions (MCQs). We believe this benchmark provides a principled and scalable testbed for studying human-like emotions, personality consistency, and value-consistent behavioural decision-making in LLM-based agents.

Related papers

• Can LLMs Truly Embody Human Personality? Analyzing AI and Human Behavior Alignment in Dispute Resolution [7.599497643290519]
  Large language models (LLMs) are increasingly used to simulate human behavior in social settings.<n>It remains unclear whether these simulations reproduce the personality-behavior patterns observed in humans.
  arXiv  Detail & Related papers  (2026-02-07T07:20:24Z)
• HUMANLLM: Benchmarking and Reinforcing LLM Anthropomorphism via Human Cognitive Patterns [59.17423586203706]
  We present HUMANLLM, a framework treating psychological patterns as interacting causal forces.<n>We construct 244 patterns from 12,000 academic papers and synthesize 11,359 scenarios where 2-5 patterns reinforce, conflict, or modulate each other.<n>Our dual-level checklists evaluate both individual pattern fidelity and emergent multi-pattern dynamics, achieving strong human alignment.
  arXiv  Detail & Related papers  (2026-01-15T08:56:53Z)
• MindShift: Analyzing Language Models' Reactions to Psychological Prompts [6.696296750931842]
  Large language models (LLMs) hold the potential to absorb and reflect personality traits and attitudes specified by users.<n>Our study introduces MindShift, a benchmark for evaluating LLMs' psychological adaptability.
  arXiv  Detail & Related papers  (2025-12-09T21:56:54Z)
• TwinVoice: A Multi-dimensional Benchmark Towards Digital Twins via LLM Persona Simulation [55.55404595177229]
  Large Language Models (LLMs) are exhibiting emergent human-like abilities.<n>TwinVoice is a benchmark for assessing persona simulation across diverse real-world contexts.
  arXiv  Detail & Related papers  (2025-10-29T14:00:42Z)
• Evaluating Personality Traits in Large Language Models: Insights from Psychological Questionnaires [3.6001840369062386]
  This work applies psychological tools to Large Language Models in diverse scenarios to generate personality profiles.<n>Our findings reveal that LLMs exhibit unique traits, varying characteristics, and distinct personality profiles even within the same family of models.
  arXiv  Detail & Related papers  (2025-02-07T16:12:52Z)
• Evaluating Large Language Models with Psychometrics [59.821829073478376]
  This paper offers a comprehensive benchmark for quantifying psychological constructs of Large Language Models (LLMs)<n>Our work identifies five key psychological constructs -- personality, values, emotional intelligence, theory of mind, and self-efficacy -- assessed through a suite of 13 datasets.<n>We uncover significant discrepancies between LLMs' self-reported traits and their response patterns in real-world scenarios, revealing complexities in their behaviors.
  arXiv  Detail & Related papers  (2024-06-25T16:09:08Z)
• PsychoGAT: A Novel Psychological Measurement Paradigm through Interactive Fiction Games with LLM Agents [68.50571379012621]
  Psychological measurement is essential for mental health, self-understanding, and personal development. PsychoGAT (Psychological Game AgenTs) achieves statistically significant excellence in psychometric metrics such as reliability, convergent validity, and discriminant validity.
  arXiv  Detail & Related papers  (2024-02-19T18:00:30Z)
• PsyCoT: Psychological Questionnaire as Powerful Chain-of-Thought for Personality Detection [50.66968526809069]
  We propose a novel personality detection method, called PsyCoT, which mimics the way individuals complete psychological questionnaires in a multi-turn dialogue manner. Our experiments demonstrate that PsyCoT significantly improves the performance and robustness of GPT-3.5 in personality detection.
  arXiv  Detail & Related papers  (2023-10-31T08:23:33Z)
• Evaluating and Inducing Personality in Pre-trained Language Models [78.19379997967191]
  We draw inspiration from psychometric studies by leveraging human personality theory as a tool for studying machine behaviors. To answer these questions, we introduce the Machine Personality Inventory (MPI) tool for studying machine behaviors. MPI follows standardized personality tests, built upon the Big Five Personality Factors (Big Five) theory and personality assessment inventories. We devise a Personality Prompting (P2) method to induce LLMs with specific personalities in a controllable way.
  arXiv  Detail & Related papers  (2022-05-20T07:32:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.