Exploring Large Language Model Agents for Piloting Social Experiments

• URL: http://arxiv.org/abs/2508.08678v1
• Date: Tue, 12 Aug 2025 06:54:13 GMT
• Title: Exploring Large Language Model Agents for Piloting Social Experiments
• Authors: Jinghua Piao, Yuwei Yan, Nian Li, Jun Zhang, Yong Li,
• Abstract summary: Computational social experiments typically employ agent-based modeling to create testbeds for piloting social experiments.<n>Despite their significance, their broader impact is largely limited by the intelligence of their core component, i.e., agents.<n>This work provides the first framework for designing LLM-driven agents to pilot social experiments.
• Score: 12.61097811161024
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Computational social experiments, which typically employ agent-based modeling to create testbeds for piloting social experiments, not only provide a computational solution to the major challenges faced by traditional experimental methods, but have also gained widespread attention in various research fields. Despite their significance, their broader impact is largely limited by the underdeveloped intelligence of their core component, i.e., agents. To address this limitation, we develop a framework grounded in well-established social science theories and practices, consisting of three key elements: (i) large language model (LLM)-driven experimental agents, serving as "silicon participants", (ii) methods for implementing various interventions or treatments, and (iii) tools for collecting behavioral, survey, and interview data. We evaluate its effectiveness by replicating three representative experiments, with results demonstrating strong alignment, both quantitatively and qualitatively, with real-world evidence. This work provides the first framework for designing LLM-driven agents to pilot social experiments, underscoring the transformative potential of LLMs and their agents in computational social science

Related papers

• Leveraging LLM-based agents for social science research: insights from citation network simulations [132.4334196445918]
  We introduce the CiteAgent framework, designed to generate citation networks based on human-behavior simulation.<n>CiteAgent captures predominant phenomena in real-world citation networks, including power-law distribution, citational distortion, and shrinking diameter.<n>We establish two LLM-based research paradigms in social science, allowing us to validate and challenge existing theories.
  arXiv  Detail & Related papers  (2025-11-05T08:47:04Z)
• Integrating LLM and Diffusion-Based Agents for Social Simulation [28.21329943306884]
  We propose a hybrid simulation framework that strategically integrates large language model (LLM)-driven agents with diffusion model-based agents.<n>Our framework outperforms existing methods in prediction accuracy, validating the effectiveness of its modular design.
  arXiv  Detail & Related papers  (2025-10-18T06:23:22Z)
• Deliberate Lab: A Platform for Real-Time Human-AI Social Experiments [9.689197691319741]
  Deliberate Lab is an open-source platform for large-scale, real-time behavioral experiments.<n>It supports both human participants and large language model (LLM)-based agents.
  arXiv  Detail & Related papers  (2025-10-14T22:02:24Z)
• Simulating Generative Social Agents via Theory-Informed Workflow Design [11.992123170134185]
  We propose a theory-informed framework that provides a systematic design process for social agents.<n>Our framework is grounded in principles from Social Cognition Theory and introduces three key modules: motivation, action planning, and learning.<n>Experiments demonstrate that our theory-driven agents reproduce realistic human behavior patterns under complex conditions.
  arXiv  Detail & Related papers  (2025-08-12T08:14:48Z)
• Dynamic Knowledge Exchange and Dual-diversity Review: Concisely Unleashing the Potential of a Multi-Agent Research Team [53.38438460574943]
  IDVSCI is a multi-agent framework built on large language models (LLMs)<n>It incorporates two key innovations: a Dynamic Knowledge Exchange mechanism and a Dual-Diversity Review paradigm.<n>Results show that IDVSCI consistently achieves the best performance across two datasets.
  arXiv  Detail & Related papers  (2025-06-23T07:12:08Z)
• MOOSE-Chem3: Toward Experiment-Guided Hypothesis Ranking via Simulated Experimental Feedback [136.27567671480156]
  We introduce experiment-guided ranking, which prioritizes hypotheses based on feedback from prior tests.<n>We frame experiment-guided ranking as a sequential decision-making problem.<n>Our approach significantly outperforms pre-experiment baselines and strong ablations.
  arXiv  Detail & Related papers  (2025-05-23T13:24:50Z)
• AgentSociety: Large-Scale Simulation of LLM-Driven Generative Agents Advances Understanding of Human Behaviors and Society [32.849311155921264]
  We propose AgentSociety, a large-scale social simulator that integrates a realistic societal environment.<n>Based on the proposed simulator, we generate social lives for over 10k agents, simulating their 5 million interactions.<n>We focus on four key social issues: polarization, the spread of inflammatory messages, the effects of universal basic income policies, and the impact of external shocks such as hurricanes.
  arXiv  Detail & Related papers  (2025-02-12T15:27:07Z)
• Designing Reliable Experiments with Generative Agent-Based Modeling: A Comprehensive Guide Using Concordia by Google DeepMind [39.96801170116895]
  Generative Agent-Based Modeling (GABM) offers a solution by enabling scholars to create simulations where AI-driven agents can generate complex behaviors. This paper introduces a framework for designing reliable experiments using GABM, making sophisticated simulation techniques more accessible to researchers across various fields.
  arXiv  Detail & Related papers  (2024-11-11T14:45:08Z)
• Unveiling and Consulting Core Experts in Retrieval-Augmented MoE-based LLMs [64.9693406713216]
  Internal mechanisms that contribute to the effectiveness of RAG systems remain underexplored. Our experiments reveal that several core groups of experts are primarily responsible for RAG-related behaviors. We propose several strategies to enhance RAG's efficiency and effectiveness through expert activation.
  arXiv  Detail & Related papers  (2024-10-20T16:08:54Z)
• Simulating Field Experiments with Large Language Models [0.6144680854063939]
  This paper pioneers the utilization of large language models (LLMs) for simulating field experiments. By introducing two novel prompting strategies, observer and participant modes, we demonstrate the ability of LLMs to both predict outcomes and replicate participant responses within complex field settings. Our findings indicate a promising alignment with actual experimental results in certain scenarios, achieving a stimulation accuracy of 66% in observer mode.
  arXiv  Detail & Related papers  (2024-08-19T03:41:43Z)
• ResearchAgent: Iterative Research Idea Generation over Scientific Literature with Large Language Models [56.08917291606421]
  ResearchAgent is an AI-based system for ideation and operationalization of novel work.<n>ResearchAgent automatically defines novel problems, proposes methods and designs experiments, while iteratively refining them.<n>We experimentally validate our ResearchAgent on scientific publications across multiple disciplines.
  arXiv  Detail & Related papers  (2024-04-11T13:36:29Z)
• Learning to Use Tools via Cooperative and Interactive Agents [58.77710337157665]
  Tool learning empowers large language models (LLMs) as agents to use external tools and extend their utility. We propose ConAgents, a Cooperative and interactive Agents framework, which coordinates three specialized agents for tool selection, tool execution, and action calibration separately. Our experiments on three datasets show that the LLMs, when equipped with ConAgents, outperform baselines with substantial improvement.
  arXiv  Detail & Related papers  (2024-03-05T15:08:16Z)
• Computational Experiments Meet Large Language Model Based Agents: A Survey and Perspective [16.08517740276261]
  Computational experiments have emerged as a valuable method for studying complex systems. accurately representing real social systems in Agent-based Modeling (ABM) is challenging due to the diverse and intricate characteristics of humans. The integration of Large Language Models (LLMs) has been proposed, enabling agents to possess anthropomorphic abilities.
  arXiv  Detail & Related papers  (2024-02-01T01:17:46Z)
• Multi-Agent Imitation Learning with Copulas [102.27052968901894]
  Multi-agent imitation learning aims to train multiple agents to perform tasks from demonstrations by learning a mapping between observations and actions. In this paper, we propose to use copula, a powerful statistical tool for capturing dependence among random variables, to explicitly model the correlation and coordination in multi-agent systems. Our proposed model is able to separately learn marginals that capture the local behavioral patterns of each individual agent, as well as a copula function that solely and fully captures the dependence structure among agents.
  arXiv  Detail & Related papers  (2021-07-10T03:49:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.