Should LLM Agents Decide in Social Simulations? Comparing Finite-State and LLM-Based Decision Policies

• URL: http://arxiv.org/abs/2606.12369v1
• Date: Wed, 10 Jun 2026 17:35:32 GMT
• Title: Should LLM Agents Decide in Social Simulations? Comparing Finite-State and LLM-Based Decision Policies
• Abstract summary: This paper evaluates whether large language models preserve an interpretable reference policy in an online social network simulation.<n>Three open-weight LLMs are tested: LLaMA 3.1, GPT-OSS, and Mistral 24B.<n>Results show that LLMs can approximate the reference policy in some configurations, but do not preserve it reliably.
• Score: 44.05636851266385
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Large language models (LLMs) are increasingly used as decision-making components in social simulations. This introduces a methodological risk: the simulation may deviate from the explicit behavioral policy defined by the researcher. In online social network (OSN) simulations, action choices shape system dynamics, interaction patterns, and model interpretability. This paper evaluates whether LLM action selectors preserve an interpretable reference policy in an OSN simulation. The reference is a finite state machine implemented as a first-order Markov model, with transition probabilities depending on the user type. The evaluation uses a synthetic network with 1,000 agents and 10,000 action decisions. Three open-weight LLMs are tested: LLaMA 3.1, GPT-OSS, and Mistral 24B. Each model is evaluated under three prompting strategies: base, guided, and probabilistic. Alignment is measured using Jensen-Shannon Divergence with Laplace smoothing, and execution time is reported. Results show that LLMs can approximate the reference policy in some configurations, but do not preserve it reliably. Alignment varies across models and prompts, and additional guidance can introduce systematic action biases. Even the best-aligned LLM configurations are several hundred times slower than direct Markov chain sampling. These findings indicate that LLM-based action selection is not a direct replacement for explicit decision policies: it can alter the intended behavior while increasing computational cost.

Related papers

• LLM-Guided Heuristic Design from Simulation Traces: A Case Study in Dynamic Production and AGV Scheduling [10.289256706393084]
  Simulation-based optimization (SBO) evaluates executable policies under dynamics.<n>Most methods treat the simulator as a black box: aggregate scores rank candidates without revealing why they fail or which policy logic should change.<n>We present an LLM-guided design framework that uses repeated simulation for selection and event-level traces for diagnosis.
  arXiv  Detail & Related papers  (2026-08-10T09:20:56Z)
• Towards Agentic Agent-based Models: Feasibility, Performance, and Statistical Model Checking [0.42025549992700273]
  Agent-based models (ABMs) rely on simple, explicit and reproducible rules for individual decision making.<n>Recent advances in large language models (LLMs) make it tempting to replace, enrich, or perturb these rules with LLM-based agentic capabilities.<n>We investigate how introducing LLM-driven decisions affect the reliability, computational cost, and behavior of ABM simulations.
  arXiv  Detail & Related papers  (2026-07-20T13:49:51Z)
• Extracting Algorithms in Pre-trained LLMs: A Case on Hidden Markov Models [12.294879009851476]
  Large language models (LLMs) display a striking ability to predict next observations from Hidden Markov Models (HMMs) via in-context learning (ICL)<n>We close this gap with a three-stage pipeline. First, we empirically compare LLM behavior against a suite of candidate algorithms and narrow the space to three classes.<n>Second, we derive theoretical connections between the three classes and show how each can be implemented in-context by a Transformer.<n>Third, returning to pre-trained LLMs, we introduce the Principal Activations Probe (PAP), a layer-wise probing and intervention method that isolates algorithmic signals in model activation
  arXiv  Detail & Related papers  (2026-06-24T17:42:19Z)
• APS: Bias-Controlled Adaptive Prototype Simulation for Population-Scale LLM Agents [23.922499684047263]
  We propose a framework that reframes scalable LLM-based simulation as a recurrent oracle-allocation problem.<n> Adaptive Prototype Simulation (APS) queries adaptive core prototypes, selected singleton-tail agents, and shadow-audit agents.<n>APS achieves a 381.1-fold reduction over full simulation, with reference-aligned final-round JSD of 0.094 against the corresponding full-LLM reference.
  arXiv  Detail & Related papers  (2026-05-19T08:45:41Z)
• Optimizing Service Operations via LLM-Powered Multi-Agent Simulation [1.2997358113120427]
  We introduce an LLM-powered multi-agent simulation framework for optimizing service operations.<n>We pose the problem as optimization with decision-dependent uncertainty.<n>We develop an on-trajectory learning algorithm that, on a single simulation run, simultaneously constructs zeroth-order gradient estimates and updates design parameters to optimize steady-state performance.
  arXiv  Detail & Related papers  (2026-04-06T03:14:33Z)
• Addressing the alignment problem in transportation policy making: an LLM approach [0.3499870393443268]
  A key challenge in transportation planning is that the collective preferences of heterogeneous travelers often diverge from the policies produced by model-driven decision tools.<n>Here, we investigate whether large language models (LLMs) can help inform and address this alignment problem.
  arXiv  Detail & Related papers  (2025-10-15T04:36:38Z)
• Reinforced Language Models for Sequential Decision Making [6.971286730860635]
  Large Language Models (LLMs) show potential as sequential decision-making agents.<n>Existing post-training methods are designed for single-turn interactions and cannot handle credit assignment in multi-step agentic tasks.<n>This work demonstrates that targeted post-training is a practical and efficient alternative to relying on model scale for creating sequential decision-making agents.
  arXiv  Detail & Related papers  (2025-08-14T17:05:44Z)
• Can Prompt Difficulty be Online Predicted for Accelerating RL Finetuning of Reasoning Models? [65.18157595903124]
  This work investigates iterative approximate evaluation for arbitrary prompts.<n>It introduces Model Predictive Prompt Selection (MoPPS), a Bayesian risk-predictive framework.<n>MoPPS reliably predicts prompt difficulty and accelerates training with significantly reduced rollouts.
  arXiv  Detail & Related papers  (2025-07-07T03:20:52Z)
• Scaling Autonomous Agents via Automatic Reward Modeling And Planning [52.39395405893965]
  Large language models (LLMs) have demonstrated remarkable capabilities across a range of tasks.<n>However, they still struggle with problems requiring multi-step decision-making and environmental feedback.<n>We propose a framework that can automatically learn a reward model from the environment without human annotations.
  arXiv  Detail & Related papers  (2025-02-17T18:49:25Z)
• LLM-Lasso: A Robust Framework for Domain-Informed Feature Selection and Regularization [59.75242204923353]
  We introduce LLM-Lasso, a framework that leverages large language models (LLMs) to guide feature selection in Lasso regression.<n>LLMs generate penalty factors for each feature, which are converted into weights for the Lasso penalty using a simple, tunable model.<n>Features identified as more relevant by the LLM receive lower penalties, increasing their likelihood of being retained in the final model.
  arXiv  Detail & Related papers  (2025-02-15T02:55:22Z)
• Analyzing Finetuning Representation Shift for Multimodal LLMs Steering [56.710375516257876]
  We propose to map hidden states to interpretable visual and textual concepts.<n>This enables us to more efficiently compare certain semantic dynamics, such as the shift from an original and fine-tuned model.<n>We also demonstrate the use of shift vectors to capture these concepts changes.
  arXiv  Detail & Related papers  (2025-01-06T13:37:13Z)
• Sense and Sensitivity: Evaluating the simulation of social dynamics via Large Language Models [27.313165173789233]
  Large language models have been proposed as a powerful replacement for classical agent-based models (ABMs) to simulate social dynamics.<n>However, due to the black box nature of LLMs, it is unclear whether LLM agents actually execute the intended semantics.<n>We show that while it is possible to engineer prompts that approximate the intended dynamics, the quality of these simulations is highly sensitive to the particular choice of prompts.
  arXiv  Detail & Related papers  (2024-12-06T14:50:01Z)
• Modular Deep Reinforcement Learning for Continuous Motion Planning with Temporal Logic [59.94347858883343]
  This paper investigates the motion planning of autonomous dynamical systems modeled by Markov decision processes (MDP) The novelty is to design an embedded product MDP (EP-MDP) between the LDGBA and the MDP. The proposed LDGBA-based reward shaping and discounting schemes for the model-free reinforcement learning (RL) only depend on the EP-MDP states.
  arXiv  Detail & Related papers  (2021-02-24T01:11:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.