Empirical Grounding Improves the Realism of LLM Agents Simulating Human Behavior During Disruptions

• URL: http://arxiv.org/abs/2607.17437v1
• Date: Sun, 19 Jul 2026 23:06:28 GMT
• Title: Empirical Grounding Improves the Realism of LLM Agents Simulating Human Behavior During Disruptions
• Abstract summary: Large language model (LLM) agents offer a generative approach to simulating human behavior under conditions that may have few or no direct historical analogues.<n>We evaluate whether empirical grounding improves the statistical realism of LLM-agent simulations during disruptions.
• Score: 4.691907035988758
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language model (LLM) agents offer a generative approach to simulating human behavior under conditions that may have few or no direct historical analogues, a common challenge in disaster and infrastructure-disruption planning. However, this generative capacity creates a validity problem: individually plausible agent reasoning may fail to reproduce empirical population behavior. We evaluate whether empirical grounding improves the statistical realism of LLM-agent simulations during disruptions. Specifically, we develop an empirically grounded LLM-agent framework that embeds demographic profiles from the American Community Survey, baseline routines from the American Time Use Survey, and urban spatial context into agent initialization, memory, decision prompts, and activity execution. An independent household survey conducted during the July 2024 Philadelphia heatwave is reserved as an external validation benchmark. Compared with an ungrounded LLM-agent baseline, the grounded model improved reconstruction of normal daily routines, increasing mean correlation with empirical activity profiles from 0.528 to 0.912 and reducing mean squared error from 0.066 to 0.008. Under heatwave conditions, the grounded model better reproduced survey-derived activity profiles, increasing mean correlation from 0.349 to 0.836 and reducing mean squared error from 0.098 to 0.012. The grounded model captured 46.4% of observed heatwave response amplitude, compared with 20.6% for the ungrounded baseline. These findings show that empirical grounding can make LLM agents more statistically credible simulators of population behavior while revealing remaining gaps in modeling human adaptation during disruptions.

Related papers

• Rethinking the Evaluation and Optimization of LLM-Based Social Simulation [56.18577997572273]
  We use the subjectivity coefficient to analyze how accuracy-based evaluation and hard-label training fail as subjectivity grows.<n>We construct SUBJSIM, a benchmark of 19,300 contexts covering 193 annotators and 100 subjective questions.<n>Results on SUBJSIM demonstrate the advantages of our method.
  arXiv  Detail & Related papers  (2026-08-20T06:26:03Z)
• "If It Looks Like a User": Measuring Real-Time Moderation Effects via Social Media Simulation [0.9134661726886931]
  We develop a calibrated extension of SimSoM, an agent-based model of information diffusion on social networks.<n>We show that the calibrated model reproduces key statistical signatures of the empirical data.<n>We apply established misinformation-spreader detection and prevention methods to both empirical and simulated data.
  arXiv  Detail & Related papers  (2026-08-17T13:58:51Z)
• Can Large Language Models Represent Urban Publics? Behavioral Replication and Population Mismatch in an Affordable-Housing Experiment [1.7984459258809926]
  We show that large language models can approximate one aggregate contrast while failing to preserve the population structure.<n>Model evaluation in urban planning should test whether this spatial and social structure survives simulation, not only average effects.
  arXiv  Detail & Related papers  (2026-07-29T16:27:36Z)
• The Epi-LLM Framework: probing LLM behavioral priors through epidemiological agent-based models [0.0]
  We introduce the Epi-LLM framework: a novel integration of agent-based modelling, real-life epigames, and large language models.<n>We find that LLM agents across four different architectures reduced peak active infections.<n>A binomial generalised linear model showed that perceived health severity was the strongest predictor of quarantine behaviour.
  arXiv  Detail & Related papers  (2026-06-01T20:31:06Z)
• The Illusion of Intervention: Your LLM-Simulated Experiment is an Observational Study [56.649987197786096]
  Large language models (LLMs) show potential as simulators of human behavior.<n>LLMs can induce unintended shifts in latent user attributes, causing user drift.<n>We formalize the confounding or selection bias that can arise due to user drift.
  arXiv  Detail & Related papers  (2026-05-20T06:09:41Z)
• Towards Real-world Human Behavior Simulation: Benchmarking Large Language Models on Long-horizon, Cross-scenario, Heterogeneous Behavior Traces [81.41397370235102]
  We introduce OmniBehavior, the first user simulation benchmark constructed entirely from real-world data.<n>We show that current models struggle to accurately simulate complex behaviors, with performance plateauing even as context windows expand.<n>This results in the loss of individual differences and long-tail behaviors, highlighting critical directions for future high-fidelity simulation research.
  arXiv  Detail & Related papers  (2026-04-09T15:26:21Z)
• RealUserSim: Bridging the Reality Gap in Agent Benchmarking via Grounded User Simulation [43.758401022409465]
  RealUserSim is the first user simulation framework grounded in real behavioral data.<n>From 14,000+ authentic human-LLM conversations (WildChat), we extract 7,275 executable behavioral profiles.<n>A fidelity benchmark (PT3) on 600 conversations across 71+ domains with anti-leakage controls shows that grounded simulation raises match rate from 24.2% to 45.3%.
  arXiv  Detail & Related papers  (2026-04-07T19:42:36Z)
• When simulations look right but causal effects go wrong: Large language models as behavioral simulators [10.754758699423022]
  Large language models (LLMs) enable researchers to specify population characteristics and intervention context in natural language.<n>We evaluated three LLMs on 11 climate-psychology interventions using a dataset of 59,508 participants from 62 countries.<n> descriptive fit did not reliably translate into causal fidelity (i.e., accurate estimates of intervention effects)
  arXiv  Detail & Related papers  (2026-04-02T18:42:29Z)
• Position: AI Agents Are Not (Yet) a Panacea for Social Simulation [62.891898598784415]
  Recent advances in large language models (LLMs) have spurred growing interest in using LLM-integrated agents for social simulation.<n>This position paper argues that LLM-based agents are not (yet) a panacea for social simulation.
  arXiv  Detail & Related papers  (2026-02-19T06:35:07Z)
• Assessing the Reliability of Persona-Conditioned LLMs as Synthetic Survey Respondents [0.4277616907160855]
  We use a large dataset of U.S. microdata to assess the impact of persona-conditioned simulations.<n>We find that persona prompting does not yield a clear aggregate improvement in survey alignment and, in many cases, significantly degrades performance.<n>Our findings highlight a key adverse impact of current persona-based simulation practices.
  arXiv  Detail & Related papers  (2026-02-06T15:13:59Z)
• Overstating Attitudes, Ignoring Networks: LLM Biases in Simulating Misinformation Susceptibility [7.616305266104683]
  Large language models (LLMs) are increasingly used as proxies for human judgment in computational social science.<n>We test whether LLM-simulated survey respondents can reproduce human patterns of misinformation belief and sharing.
  arXiv  Detail & Related papers  (2026-02-04T15:48:05Z)
• MF-LLM: Simulating Population Decision Dynamics via a Mean-Field Large Language Model Framework [53.82097200295448]
  Mean-Field LLM (MF-LLM) is first to incorporate mean field theory into social simulation.<n>MF-LLM models bidirectional interactions between individuals and the population through an iterative process.<n> IB-Tune is a novel fine-tuning method inspired by the Information Bottleneck principle.
  arXiv  Detail & Related papers  (2025-04-30T12:41:51Z)
• GenSim: A General Social Simulation Platform with Large Language Model based Agents [111.00666003559324]
  We propose a novel large language model (LLMs)-based simulation platform called textitGenSim.<n>Our platform supports one hundred thousand agents to better simulate large-scale populations in real-world contexts.<n>To our knowledge, GenSim represents an initial step toward a general, large-scale, and correctable social simulation platform.
  arXiv  Detail & Related papers  (2024-10-06T05:02:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.