OdysSim: Building Foundation Models for Human Behavior Simulation
- URL: http://arxiv.org/abs/2606.14199v1
- Date: Fri, 12 Jun 2026 07:31:55 GMT
- Title: OdysSim: Building Foundation Models for Human Behavior Simulation
- Abstract summary: Large language models are increasingly deployed as human simulators for interactive evaluation and social simulation.<n>Yet helpfulness-driven post-training pulls them toward a homogeneous, overly agreeable assistant register.<n>We present OdysSim, the largest open systematic investigation of behavioral foundation models.
- Score: 70.35265860287608
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Large language models are increasingly deployed as human simulators for interactive evaluation and social simulation. Yet helpfulness-driven post-training pulls them toward a homogeneous, overly agreeable assistant register, creating a behavioral Sim2Real gap. We present OdysSim, the largest open systematic investigation of behavioral foundation models, i.e., models trained to simulate human behavior at scale. We propose SOUL, a taxonomy of five capability axes (CONV, SS, COG, ROLE, EVAL) that unifies 62 datasets and 23 benchmark tasks under one framework. Specifically, we curate the OdysSim corpus (21.4M interactions, 10B tokens, retrofitted with back-generated social contexts), construct the SOUL-Index benchmark, and develop an end-to-end training recipe combining midtraining, task-specific RL, and expert distillation. The resulting open 8B OSim model ranks first or tied-first on 8 of 23 tasks, outperforming any individual frontier model by this count, with the strongest gains on conversational and social tasks. Its outputs are also more human-like in length, formatting, and word choice, and it transfers zero-shot to out-of-distribution user simulation on $τ$-bench, nearly matching real users on reaction alignment (93.2 vs. 93.5). We further show that LLM-as-judge RL induces reward-hacking patterns, and that our detectors can mitigate them during post-training. Together, our findings suggest that behavioral foundation models require rethinking the LLM training paradigm. We release all artifacts to support future research.
Related papers
- Will Scaling Improve Social Simulation with LLMs? [87.15677770205751]
Large Language Model (LLM) social simulations are a promising research method, but they are not yet faithful enough to be adopted widely.<n>We use scaling laws to study the relationship between LLMs' compute scale, general capability benchmarks, and the fidelity of social simulation.
arXiv Detail & Related papers (2026-07-02T17:30:38Z) - Reinforcing Human Behavior Simulation via Verbal Feedback [105.68319269895653]
We present DITTO, a model trained by treating verbal feedback as a first-class signal in reinforcement learning.<n>We also introduce SOUL, a unified benchmark and training data suite spanning 10 tasks across six categories: Theory of Mind, character role play, social skill, learner simulation, user simulation, and persona simulation.<n>DitTO achieves an average 36% improvement over the base model and exceeds GPT-5.4 on 6 of 10 SOUL benchmarks.
arXiv Detail & Related papers (2026-05-19T21:23:24Z) - Mind the Sim2Real Gap in User Simulation for Agentic Tasks [101.69142591891234]
We formalize the Sim2Real gap in user simulation and present the first study running the full $$-bench protocol with real humans.<n> Behaviorally, LLM simulators are excessively cooperative, stylistically uniform, and lack realistic frustration or ambiguity.<n>These findings highlight the importance of human validation when using LLM-based user simulators in the agent development cycle.
arXiv Detail & Related papers (2026-03-11T19:12:31Z) - SimBench: Benchmarking the Ability of Large Language Models to Simulate Human Behaviors [58.87134689752605]
We introduce SimBench, the first large-scale, standardized benchmark for a robust, reproducible science of LLM simulation.<n>We show that even the best LLMs today have limited simulation ability (score: 40.80/100), performance scales log-linearly with model size.<n>We demonstrate that simulation ability correlates most strongly with deep, knowledge-intensive reasoning.
arXiv Detail & Related papers (2025-10-20T13:14:38Z) - Shop-R1: Rewarding LLMs to Simulate Human Behavior in Online Shopping via Reinforcement Learning [27.226155951073064]
Shop-R1 is a novel reinforcement learning framework aimed at enhancing the reasoning ability of Large Language Models (LLMs)<n>For rationale generation, we leverage internal model signals (e.g., logit distributions) to guide the reasoning process in a self-supervised manner.<n>For action prediction, we propose a hierarchical reward structure with difficulty-aware scaling to prevent reward hacking.
arXiv Detail & Related papers (2025-07-23T18:10:43Z) - YuLan-OneSim: Towards the Next Generation of Social Simulator with Large Language Models [50.35333054932747]
We introduce a novel social simulator called YuLan-OneSim.<n>Users can simply describe and refine their simulation scenarios through natural language interactions with our simulator.<n>We implement 50 default simulation scenarios spanning 8 domains, including economics, sociology, politics, psychology, organization, demographics, law, and communication.
arXiv Detail & Related papers (2025-05-12T14:05:17Z) - Sparse Rewards Can Self-Train Dialogue Agents [22.799506097310008]
We introduce a novel self-improvement paradigm that empowers LLM agents to autonomously enhance their performance without external human feedback.<n>We present ToolWOZ, a sparse reward tool-calling simulation environment derived from MultiWOZ.<n>We demonstrate that models trained with JOSH, both small and frontier, significantly improve tool-based interactions while preserving general model capabilities across diverse benchmarks.
arXiv Detail & Related papers (2024-09-06T21:00:57Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.