Benchmarking LLMs for Community Governance Simulation with Life-history Narratives

• URL: http://arxiv.org/abs/2605.23783v1
• Date: Fri, 22 May 2026 15:48:49 GMT
• Title: Benchmarking LLMs for Community Governance Simulation with Life-history Narratives
• Abstract summary: Large language models (LLMs) are scalable to simulate human attitudes and behaviors at low cost.<n>We present an integrated research framework covering dataset, benchmark, algorithm, and system.<n>System integrates curriculum-LoRA into a closed-loop policy-evaluation pipeline.
• Score: 46.86050402684712
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Effective community governance hinges on understanding what specific residents think and need. Recent work has used large language models (LLMs) to simulate human respondents, offering a scalable, reproducible way to study human attitudes and behaviors at low cost. However, these studies typically prompt the model with just a few demographic variables (age, gender, income), simulating only general role types. This is insufficient for community governance, where decisions depend on the views of specific residents. We bridge this gap with an integrated research framework covering dataset, benchmark, algorithm, and system. The dataset comprises approximately 1.2 million characters of first-person narrative collected through two-hour semi-structured interviews with each of 92 residents in an urban community, organized around nine community-governance domains. The benchmark probes 18 mainstream LLMs across four prompting strategies and shows that adding rich life-history profiles meaningfully raises fidelity above the no-profile baseline, but this gain comes with more input tokens per call from the longer prompts they require. The algorithm, curriculum-LoRA, is a parameter-efficient personalization framework that, by closing this fidelity-cost gap, matches the strongest baseline's fidelity at roughly 10x lower per-call cost and Pareto-dominates every configuration tested. The system integrates curriculum-LoRA into a closed-loop policy-evaluation pipeline. Together, these results bring individual-level LLM-based resident simulation within reach of resource-constrained local administrations, enabling community-governance decisions to be systematically pre-evaluated in silico before real-world deployment.

Related papers

• SpatialWorld: Benchmarking Interactive Spatial Reasoning of Multimodal Agents in Real-World Tasks [73.92333717662558]
  SpatialWorld is a benchmark for evaluating the interactive spatial understanding of multimodal agents in complex real-world tasks.<n>It features 760 human-annotated tasks across diverse domains (e.g., household routines, travel, social collaboration)<n>For reliable evaluation, each task includes a human-validated initial state, a reference trajectory, and a terminal-state verifier.
  arXiv  Detail & Related papers  (2026-06-08T15:51:51Z)
• LiveCultureBench: a Multi-Agent, Multi-Cultural Benchmark for Large Language Models in Dynamic Social Simulations [63.478832978278014]
  Large language models (LLMs) are increasingly deployed as autonomous agents, yet evaluations focus primarily on task success rather than cultural appropriateness or evaluator reliability.<n>We introduce LiveCultureBench, a multi-cultural, dynamic benchmark that embeds LLMs as agents in a simulated town and evaluates them on both task completion and adherence to socio-cultural norms.
  arXiv  Detail & Related papers  (2026-03-02T15:04:16Z)
• Individual Turing Test: A Case Study of LLM-based Simulation Using Longitudinal Personal Data [54.145424717168794]
  Large Language Models (LLMs) have demonstrated remarkable human-like capabilities, yet their ability to replicate a specific individual remains under-explored.<n>This paper presents a case study to investigate LLM-based individual simulation with a volunteer-contributed archive of private messaging history spanning over ten years.<n>We propose the "Individual Turing Test" to evaluate whether acquaintances of the volunteer can correctly identify which response in a multi-candidate pool most plausibly comes from the volunteer.
  arXiv  Detail & Related papers  (2026-03-01T21:46:27Z)
• SocioBench: Modeling Human Behavior in Sociological Surveys with Large Language Models [32.66051406264919]
  Large language models (LLMs) show strong potential for simulating human social behaviors and interactions, yet lack large-scale, systematically constructed benchmarks for evaluating their alignment with real-world social attitudes.<n>We introduce SocioBench-a comprehensive benchmark derived from the annually collected, standardized survey data of the International Social Survey Programme (ISSP)<n>The benchmark aggregates over 480,000 real respondent records from more than 30 countries, spanning 10 sociological domains and over 40 demographic attributes.
  arXiv  Detail & Related papers  (2025-10-13T08:22:20Z)
• Population-Aligned Persona Generation for LLM-based Social Simulation [58.84363795421489]
  We propose a systematic framework for synthesizing high-quality, population-aligned persona sets for social simulation.<n>Our approach begins by leveraging large language models to generate narrative personas from long-term social media data.<n>To address the needs of specific simulation contexts, we introduce a task-specific module that adapts the globally aligned persona set to targeted subpopulations.
  arXiv  Detail & Related papers  (2025-09-12T10:43:47Z)
• Large Language Models as Virtual Survey Respondents: Evaluating Sociodemographic Response Generation [18.225151370273093]
  This paper explores a new paradigm: simulating virtual survey respondents using Large Language Models (LLMs)<n>We introduce two novel simulation settings, namely Partial Attribute Simulation (PAS) and Full Attribute Simulation (FAS)<n>We curate a comprehensive benchmark suite, LLM-S3 (Large Language Model-based Sociodemographic Simulation Survey), that spans 11 real-world public datasets across four sociological domains.
  arXiv  Detail & Related papers  (2025-09-08T04:59:00Z)
• MF-LLM: Simulating Population Decision Dynamics via a Mean-Field Large Language Model Framework [53.82097200295448]
  Mean-Field LLM (MF-LLM) is first to incorporate mean field theory into social simulation.<n>MF-LLM models bidirectional interactions between individuals and the population through an iterative process.<n> IB-Tune is a novel fine-tuning method inspired by the Information Bottleneck principle.
  arXiv  Detail & Related papers  (2025-04-30T12:41:51Z)
• LLM Generated Persona is a Promise with a Catch [18.45442859688198]
  Persona-based simulations hold promise for transforming disciplines that rely on population-level feedback.<n>Traditional methods to collect realistic persona data face challenges.<n>They are prohibitively expensive and logistically challenging due to privacy constraints.
  arXiv  Detail & Related papers  (2025-03-18T03:11:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.