Benchmarking large language model agent societies against human behavioural distributions

• URL: http://arxiv.org/abs/2608.28182v1
• Date: Fri, 28 Aug 2026 10:50:32 GMT
• Title: Benchmarking large language model agent societies against human behavioural distributions
• Abstract summary: Populations of large language model agents are increasingly used as experimental societies.<n>Three doubts shadow every such result: whether the agents behave like the humans they stand in for, whether a finding survives changes to the apparatus that leave the rules untouched, and whether apparent social dynamics are interaction at all rather than the reproduction of experiments the models have read.<n>This article introduces SILICA, an open instrument that tests all three.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Populations of large language model agents are increasingly used as experimental societies. Three doubts shadow every such result: whether the agents behave like the humans they stand in for, whether a finding survives changes to the apparatus that leave the rules untouched, and whether apparent social dynamics are interaction at all rather than the reproduction of experiments the models have read. This article introduces SILICA, an open instrument that tests all three. Five environments carry published human anchors, each paired with perturbations that re-render the same rules and with variants whose payoffs point away from the memorised result. Twelve open-weight models were run through it on a single consumer graphics card. Agreement with human data is confined to starting points: first-round public-goods contributions fall inside the equivalence margin for eight of eleven models, while no model matches end-state contributions or the human corridor of cooperation. Merely swapping the order in which two actions are listed costs one model 58 points of cooperation. Presenting responders with a fixed schedule of offers shows that only one model, the sole reasoning-trained one, places its acceptance threshold where the incentive requires; two move theirs part of the way, two move them the wrong way, and three never acquire one. Conventions form through a shared prior over the names rather than through negotiation, though negotiation reappears once that prior is disrupted. On the certification ladder defined here, current silicon societies support exploratory claims and no more.

Related papers

• The average-farmer illusion in language-model simulations of agricultural decisions [9.14716652210035]
  Language-model agents are increasingly used as synthetic people in surveys and social simulations.<n>We compare Claude, Codex and Kimi under four prespecified prompt designs with matched farmer decisions from China and four African countries.
  arXiv  Detail & Related papers  (2026-09-14T04:54:51Z)
• Matched Outcomes, Divergent Gaze: How Foveated MLLMs Search Compared to Humans [50.71222402701153]
  We compare three general-purpose large language models (MLLMs) with human eye-movement scanpaths on goal-directed search (COCO-Search18)<n>All three share one signature: low-entropy, large-amplitude, self-consistent scanpaths that agree with themselves far more closely than two humans agree with each other.<n>The gap sits on a process axis that answer-alignment and saliency metrics do not measure. Because they miss it, such metrics cannot certify human-like vision, and zero-shot models suit outcome and spatial questions but not temporal, process-level
  arXiv  Detail & Related papers  (2026-08-17T12:54:39Z)
• When Agents Lie: Premeditation, Persistence, and Exploitation in Repeated Games [69.060029640261]
  We evaluate three frontier models across six games in homogeneous and heterogeneous groups over 10 rounds.<n>Different models interpret announcements incompatibly, some as binding commitments and others as cheap talk.
  arXiv  Detail & Related papers  (2026-07-06T14:17:59Z)
• GrowLoop: Self-Evolving Conversation Evaluation Seeded by Human [7.998927393247182]
  We propose GrowLoop, a self-evolving evaluation system that continuously adapts as models advance and scenarios shift.<n>Our work shifts the paradigm from manual updates or difficulty scaling to comprehensive, continuous self-evolution.
  arXiv  Detail & Related papers  (2026-05-26T16:53:14Z)
• Agentic World Modeling: Foundations, Capabilities, Laws, and Beyond [209.35045331678043]
  We introduce a "levels x laws" taxonomy organized along two axes.<n>The first defines three capability levels: L1 Predictor, which learns one-step local transition operators; L2 Simulator, which composes them into multi-step, action-conditioned rollouts that respect domain laws; and L3 Evolver, which autonomously revises its own model when predictions fail against new evidence.<n>We synthesize over 400 works and summarize more than 100 representative systems spanning model-based reinforcement learning, video generation, web and GUI agents, multi-agent social simulation, and AI-driven scientific discovery.
  arXiv  Detail & Related papers  (2026-04-24T17:48:47Z)
• Beyond Preset Identities: How Agents Form Stances and Boundaries in Generative Societies [28.436766185842767]
  This paper proposes a novel mixed-methods framework combining computational virtual ethnography with quantitative socio-cognitive profiling.<n>It formalizes three new metrics: Innate Value Bias (IVB), Persuasion Sensitivity, and Trust-Action Decoupling (TAD)<n>Findings expose the fragility of static prompt engineering, providing a methodological and quantitative foundation for dynamic alignment in human-agent hybrid societies.
  arXiv  Detail & Related papers  (2026-03-24T16:38:46Z)
• Do people expect different behavior from large language models acting on their behalf? Evidence from norm elicitations in two canonical economic games [0.0]
  We study whether people have different expectations when decisions are made by large language models acting on their behalf instead of themselves.<n>We use the Krupka-Weber norm elicitation task to detect shifts in social appropriateness ratings.<n>Results suggest that people apply different norms for machines deciding on how to split resources but are not opposed to machines enforcing the norms.
  arXiv  Detail & Related papers  (2026-01-14T16:07:00Z)
• Multi-Level Collaboration in Model Merging [56.31088116526825]
  This paper explores the intrinsic connections between model merging and model ensembling.<n>We find that even when previous restrictions are not met, there is still a way for model merging to attain a near-identical and superior performance similar to that of ensembling.
  arXiv  Detail & Related papers  (2025-03-03T07:45:04Z)
• EgoAgent: A Joint Predictive Agent Model in Egocentric Worlds [119.02266432167085]
  We propose EgoAgent, a unified agent model that simultaneously learns to represent, predict, and act within a single transformer.<n>EgoAgent explicitly models the causal and temporal dependencies among these abilities by formulating the task as an interleaved sequence of states and actions.<n> Comprehensive evaluations of EgoAgent on representative tasks such as image classification, egocentric future state prediction, and 3D human motion prediction demonstrate the superiority of our method.
  arXiv  Detail & Related papers  (2025-02-09T11:28:57Z)
• Predictive Churn with the Set of Good Models [61.00058053669447]
  This paper explores connections between two seemingly unrelated concepts of predictive inconsistency.<n>The first, known as predictive multiplicity, occurs when models that perform similarly produce conflicting predictions for individual samples.<n>The second concept, predictive churn, examines the differences in individual predictions before and after model updates.
  arXiv  Detail & Related papers  (2024-02-12T16:15:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.