When Reasoning Models Hurt Behavioral Simulation: A Solver-Sampler Mismatch in Multi-Agent LLM Negotiation

• URL: http://arxiv.org/abs/2604.11840v1
• Date: Sun, 12 Apr 2026 13:36:10 GMT
• Title: When Reasoning Models Hurt Behavioral Simulation: A Solver-Sampler Mismatch in Multi-Agent LLM Negotiation
• Abstract summary: A common assumption is that stronger reasoning should improve simulation fidelity.<n>We argue that this assumption can fail when the objective is not to solve a strategic problem, but to sample plausible boundedly rational behavior.<n>We study this solver-sampler mismatch in three multi-agent negotiation environments adapted from earlier simulation work.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models are increasingly used as agents in social, economic, and policy simulations. A common assumption is that stronger reasoning should improve simulation fidelity. We argue that this assumption can fail when the objective is not to solve a strategic problem, but to sample plausible boundedly rational behavior. In such settings, reasoning-enhanced models can become better solvers and worse simulators: they can over-optimize for strategically dominant actions, collapse compromise-oriented terminal behavior, and sometimes exhibit a diversity-without-fidelity pattern in which local variation survives without outcome-level fidelity. We study this solver-sampler mismatch in three multi-agent negotiation environments adapted from earlier simulation work: an ambiguous fragmented-authority trading-limits scenario, an ambiguous unified-opposition trading-limits scenario, and a new-domain grid-curtailment case in emergency electricity management. We compare three reflection conditions, no reflection, bounded reflection, and native reasoning, across two primary model families and then extend the same protocol to direct OpenAI runs with GPT-4.1 and GPT-5.2. Across all three experiments, bounded reflection produces substantially more diverse and compromise-oriented trajectories than either no reflection or native reasoning. In the direct OpenAI extension, GPT-5.2 native ends in authority decisions in 45 of 45 runs across the three experiments, while GPT-5.2 bounded recovers compromise outcomes in every environment. The contribution is not a claim that reasoning is generally harmful. It is a methodological warning: model capability and simulation fidelity are different objectives, and behavioral simulation should qualify models as samplers, not only as solvers.

Related papers

• One Frozen Simulator Is Not Enough: Simulator Collapse in Multi-Agent RL [75.34038228759182]
  Multi-agent reinforcement learning for human-AI interaction typically relies on a single large language model to simulate user behavior.<n>We show that this approach systematically fails to generalize, and trace the failure to simulator collapse.<n>We propose two complementary solutions, one at inference time and one at training time.
  arXiv  Detail & Related papers  (2026-08-12T16:55:50Z)
• Divergence Meets Consensus: A Multi-Source Negative Sampling Framework for Sequential Recommendation [12.425187572716574]
  "Teacher-Peer-Self" framework is inspired by Vygotsky's Zone of Proximal Development (ZPD) theory.<n>It incorporates multi-source scoring, divergence re-ranking, and consensus distillation.<n>It consistently outperforms state-of-the-art negative sampling methods.
  arXiv  Detail & Related papers  (2026-05-19T10:42:23Z)
• Masked Diffusion Language Models are Strong and Steerable Text-Based World Models for Agentic RL [0.8546394944138487]
  We formalize text-based world modeling as a steerable transition-dynamics problem decomposed into initial state.<n>We curate 239,403 grounded state-action trajectories spanning nine open-source environments and twelve frontier model families.<n>We introduce a plug-and-play GRPO training framework with deterministic state checks, and perform zero-shot transfer ablations on three OOD environments.
  arXiv  Detail & Related papers  (2026-05-07T00:40:32Z)
• Agentic World Modeling: Foundations, Capabilities, Laws, and Beyond [209.35045331678043]
  We introduce a "levels x laws" taxonomy organized along two axes.<n>The first defines three capability levels: L1 Predictor, which learns one-step local transition operators; L2 Simulator, which composes them into multi-step, action-conditioned rollouts that respect domain laws; and L3 Evolver, which autonomously revises its own model when predictions fail against new evidence.<n>We synthesize over 400 works and summarize more than 100 representative systems spanning model-based reinforcement learning, video generation, web and GUI agents, multi-agent social simulation, and AI-driven scientific discovery.
  arXiv  Detail & Related papers  (2026-04-24T17:48:47Z)
• Too Correct to Learn: Reinforcement Learning on Saturated Reasoning Data [55.84428098924793]
  We propose Constrained Uniform Top-K Sampling (CUTS), a parameter-free decoding strategy enforcing structure-preserving exploration.<n>We integrate this into Mixed-CUTS, a training framework synergizing exploitative and exploratory rollouts to amplify intra-group advantage variance.<n> Notably, Mixed-CUTS improves Pass@1 accuracy on the challenging AIME25 benchmark by up to 15.1% over standard GRPO.
  arXiv  Detail & Related papers  (2026-04-20T16:43:28Z)
• Meituan Merchant Business Diagnosis via Policy-Guided Dual-Process User Simulation [15.639936579155922]
  Simulating group-level user behavior enables scalable counterfactual evaluation of merchant strategies without costly online experiments.<n>We propose Policy-Guided Hybrid Simulation (PGHS), a dual-process framework that mines transferable decision policies from behavioral trajectories.<n>PGHS achieves a group simulation error of 8.80%, improving over the best reasoning-based and fitting-based baselines by 45.8% and 40.9% respectively.
  arXiv  Detail & Related papers  (2026-04-16T16:23:13Z)
• Towards Real-world Human Behavior Simulation: Benchmarking Large Language Models on Long-horizon, Cross-scenario, Heterogeneous Behavior Traces [81.41397370235102]
  We introduce OmniBehavior, the first user simulation benchmark constructed entirely from real-world data.<n>We show that current models struggle to accurately simulate complex behaviors, with performance plateauing even as context windows expand.<n>This results in the loss of individual differences and long-tail behaviors, highlighting critical directions for future high-fidelity simulation research.
  arXiv  Detail & Related papers  (2026-04-09T15:26:21Z)
• Amortized Inference for Correlated Discrete Choice Models via Equivariant Neural Networks [0.0]
  We propose an amortized approach using a neural network emulator to approximate choice probabilities for general error distributions.<n>We use Sobolev training, augmenting the likelihood loss with a gradient-matching penalty so that the emulator learns both choice probabilities and their derivatives.
  arXiv  Detail & Related papers  (2026-03-25T18:30:11Z)
• Sim2Act: Robust Simulation-to-Decision Learning via Adversarial Calibration and Group-Relative Perturbation [54.29523408543184]
  Simulation-to-decision learning enables safe policy training in digital environments without risking real-world deployment.<n>Existing approaches either focus on improving average simulation fidelity or adopt conservative regularization.<n>We propose Sim2Act, a robust simulation-to-decision framework that addresses both simulator and policy robustness.
  arXiv  Detail & Related papers  (2026-03-10T00:51:47Z)
• AutoControl Arena: Synthesizing Executable Test Environments for Frontier AI Risk Evaluation [22.833977971686966]
  We present AutoControl Arena, an automated framework for frontier AI risk evaluation.<n>We mitigate hallucination while maintaining flexibility.<n>This principle, instantiated through a three-agent framework, achieves over 98% end-to-end success and 60% human preference over existing simulators.
  arXiv  Detail & Related papers  (2026-03-08T02:49:45Z)
• CODE: A Contradiction-Based Deliberation Extension Framework for Overthinking Attacks on Retrieval-Augmented Generation [43.85448261466922]
  We propose an end-to-end attack framework named Contradiction-Based Deliberation Extension (CODE)<n> CODE develops a multi-agent architecture to construct poisoning samples that are injected into the knowledge base.<n>Experiments show that CODE causes a 5.32x-24.72x increase in reasoning token consumption, without degrading task performance.
  arXiv  Detail & Related papers  (2026-01-19T14:52:31Z)
• Adversarial Yet Cooperative: Multi-Perspective Reasoning in Retrieved-Augmented Language Models [72.4149653187766]
  We propose a Reasoner-Verifier framework named Adrialversa Reasoning RAG (ARR)<n>The Reasoner and Verifier engage in reasoning on retrieved evidence and critiquing each other's logic while being guided by process-aware advantage.<n> Experiments on multiple benchmarks demonstrate the effectiveness of our method.
  arXiv  Detail & Related papers  (2026-01-08T06:57:03Z)
• Robust Optimization with Diffusion Models for Green Security [49.68562792424776]
  In green security, defenders must forecast adversarial behavior, such as poaching, illegal logging, and illegal fishing, to plan effective patrols.<n>We propose a conditional diffusion model for adversary behavior modeling, leveraging its strong distribution-fitting capabilities.<n>We introduce a mixed strategy of mixed strategies and employ a twisted Sequential Monte Carlo (SMC) sampler for accurate sampling.
  arXiv  Detail & Related papers  (2025-02-19T05:30:46Z)
• SAFE-SIM: Safety-Critical Closed-Loop Traffic Simulation with Diffusion-Controllable Adversaries [94.84458417662407]
  We introduce SAFE-SIM, a controllable closed-loop safety-critical simulation framework. Our approach yields two distinct advantages: 1) generating realistic long-tail safety-critical scenarios that closely reflect real-world conditions, and 2) providing controllable adversarial behavior for more comprehensive and interactive evaluations. We validate our framework empirically using the nuScenes and nuPlan datasets across multiple planners, demonstrating improvements in both realism and controllability.
  arXiv  Detail & Related papers  (2023-12-31T04:14:43Z)
• Policy Learning for Robust Markov Decision Process with a Mismatched Generative Model [42.28001762749647]
  In high-stake scenarios like medical treatment and auto-piloting, it's risky or even infeasible to collect online experimental data to train the agent. We consider policy learning for Robust Markov Decision Processes (RMDP), where the agent tries to seek a robust policy with respect to unexpected perturbations on the environments. Our goal is to identify a near-optimal robust policy for the perturbed testing environment, which introduces additional technical difficulties.
  arXiv  Detail & Related papers  (2022-03-13T06:37:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.