Sparks of In Silico Cognitive Science: Theories from Simulated Data Can Generalize to Humans

• URL: http://arxiv.org/abs/2609.08003v1
• Date: Mon, 07 Sep 2026 21:35:50 GMT
• Title: Sparks of In Silico Cognitive Science: Theories from Simulated Data Can Generalize to Humans
• Abstract summary: Behavioral foundation models have been proposed as stand-ins for human participants across settings.<n>It is unclear whether theories discovered on them generalize to humans or merely characterize the simulator.<n>We run a closed-loop discovery system in which agents design experiments, collect responses, arbitrate between competing theories, and synthesize successors.
• Score: 27.855847876496238
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Behavioral foundation models have been proposed as stand-ins for human participants across settings, but it is unclear whether theories discovered on them generalize to humans or merely characterize the simulator. We ran the Automated Cognitive Scientist (\textsc{AutoCog}), a closed-loop discovery system in which LLM agents design theory-discriminating experiments, collect responses, arbitrate between competing theories, and synthesize successors, entirely on behavior simulated by Centaur, a foundation model of human behavior. In a multi-attribute decision-making setting, the theories \textsc{AutoCog} found on Centaur generalized to human data: they outperformed canonical theories on ten held-out experiments and were rivaled only by theories found by running the same loop on people. We argue that this succeeds despite the simulator's inevitable imperfections because a discovery loop that arbitrates between competing theories demands less of its simulator than estimation does. The simulator only needs to capture the regularities that distinguish the theories, and not necessarily reproduce behavior precisely. Imperfect simulators can therefore widen the search over theories, with human data then testing whether the surfaced theories generalize.

Related papers

• auto-psych: Automating the science of mind using agent-driven theory discovery and experimentation [30.489563356452255]
  We apply automated discovery techniques to the project of generating theories in computational cognitive science.<n>We use a classic case study from cognitive psychology: judging which sequences of coin flips seem subjectively more random.<n>Our system, auto-psych, uses nested agent-based discovery loops to generate explanatory theories of human behavior.
  arXiv  Detail & Related papers  (2026-06-24T23:43:01Z)
• Closing the Loop to Discover Psychological Theories with an Automated Cognitive Scientist [26.846270187624636]
  We present the Automated Cognitive Scientist (AutoCog), a fully autonomous agentic-AI system.<n>We show how it can be used to turn cognitive theory-building into an explicit, executable, and cumulative science.
  arXiv  Detail & Related papers  (2026-06-24T23:20:58Z)
• Automated Adversarial Collaboration for Advancing Theory Building in the Cognitive Sciences [9.858210301644549]
  We introduce an automated adversarial collaboration framework for adjudicating among competing theories.<n>The system combines LLM-based theory agents, program synthesis, and information-theoretic experimental design in a closed loop.
  arXiv  Detail & Related papers  (2026-04-28T11:41:08Z)
• Towards Real-world Human Behavior Simulation: Benchmarking Large Language Models on Long-horizon, Cross-scenario, Heterogeneous Behavior Traces [81.41397370235102]
  We introduce OmniBehavior, the first user simulation benchmark constructed entirely from real-world data.<n>We show that current models struggle to accurately simulate complex behaviors, with performance plateauing even as context windows expand.<n>This results in the loss of individual differences and long-tail behaviors, highlighting critical directions for future high-fidelity simulation research.
  arXiv  Detail & Related papers  (2026-04-09T15:26:21Z)
• Large language models replicate and predict human cooperation across experiments in game theory [0.8166364251367626]
  How closely large language models mirror actual human decision-making remains poorly understood.<n>We develop a digital twin of game-theoretic experiments and introduce a systematic prompting and probing framework for machine-behavioral evaluation.<n>We find that Llama reproduces human cooperation patterns with high fidelity, capturing human deviations from rational choice theory.
  arXiv  Detail & Related papers  (2025-11-06T16:21:27Z)
• The Universal Landscape of Human Reasoning [60.72403709545137]
  We introduce Information Flow Tracking (IF-Track) to quantify information entropy and gain at each reasoning step.<n>We show that IF-Track captures essential reasoning features, identifies systematic error patterns, and characterizes individual differences.<n>This approach establishes a quantitative bridge between theory and measurement, offering mechanistic insights into the architecture of reasoning.
  arXiv  Detail & Related papers  (2025-10-24T16:26:36Z)
• SimBench: Benchmarking the Ability of Large Language Models to Simulate Human Behaviors [58.87134689752605]
  We introduce SimBench, the first large-scale, standardized benchmark for a robust, reproducible science of LLM simulation.<n>We show that even the best LLMs today have limited simulation ability (score: 40.80/100), performance scales log-linearly with model size.<n>We demonstrate that simulation ability correlates most strongly with deep, knowledge-intensive reasoning.
  arXiv  Detail & Related papers  (2025-10-20T13:14:38Z)
• PolySim: Bridging the Sim-to-Real Gap for Humanoid Control via Multi-Simulator Dynamics Randomization [53.7088694598817]
  We introduce PolySim, a WBC training platform that integrates multiple heterogeneous simulators.<n>Theoretically, we show that PolySim yields a tighter upper bound on simulator inductive bias than single-simulator training.
  arXiv  Detail & Related papers  (2025-10-02T06:31:42Z)
• Hypothesis-Driven Theory-of-Mind Reasoning for Large Language Models [76.6028674686018]
  We introduce thought-tracing, an inference-time reasoning algorithm to trace the mental states of agents.<n>Our algorithm is modeled after the Bayesian theory-of-mind framework.<n>We evaluate thought-tracing on diverse theory-of-mind benchmarks, demonstrating significant performance improvements.
  arXiv  Detail & Related papers  (2025-02-17T15:08:50Z)
• User Behavior Simulation with Large Language Model based Agents [116.74368915420065]
  We propose an LLM-based agent framework and design a sandbox environment to simulate real user behaviors. Based on extensive experiments, we find that the simulated behaviors of our method are very close to the ones of real humans.
  arXiv  Detail & Related papers  (2023-06-05T02:58:35Z)
• JECC: Commonsense Reasoning Tasks Derived from Interactive Fictions [75.42526766746515]
  We propose a new commonsense reasoning dataset based on human's Interactive Fiction (IF) gameplay walkthroughs. Our dataset focuses on the assessment of functional commonsense knowledge rules rather than factual knowledge. Experiments show that the introduced dataset is challenging to previous machine reading models as well as the new large language models.
  arXiv  Detail & Related papers  (2022-10-18T19:20:53Z)
• A consolidating review of Spekkens' toy theory [0.0]
  Spekkens' toy theory is based off a simple premise: "What if we took a common classical theory and added the uncertainty principle as a postulate?" We consolidate different approaches to Spekkens' toy theory, including the stabilizer formalism and the generalization to arbitrary dimensions.
  arXiv  Detail & Related papers  (2021-05-07T14:10:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.