Quantifying the Utility of User Simulators for Building Collaborative LLM Assistants

• URL: http://arxiv.org/abs/2605.09808v1
• Date: Sun, 10 May 2026 23:06:24 GMT
• Title: Quantifying the Utility of User Simulators for Building Collaborative LLM Assistants
• Abstract summary: We show how simulator quality can be quantified in terms of its downstream utility.<n>We train LLM assistants via reinforcement learning against a spectrum of simulators.<n>As evaluation, we measure pairwise win rates in a user study with 283 participants.
• Score: 7.523995265564992
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: User simulators are increasingly leveraged to build interactive AI assistants, yet how to measure the quality of these simulators remains an open question. In this work, we show how simulator quality can be quantified in terms of its downstream utility: how an LLM assistant trained with this user simulator performs in the wild when interacting with real humans. In a controlled experiment where only the user simulator varies, we train LLM assistants via reinforcement learning against a spectrum of simulators, from an LLM prompted to role-play a user to one fine-tuned on human utterances from WildChat. As evaluation, we measure pairwise win rates in a user study with 283 participants and on WildBench, a benchmark derived from real human--AI conversations. Training against the role-playing LLM yields an assistant statistically indistinguishable from the initial assistant in our user study (51% win rate), whereas training against the fine-tuned simulator yields significant gains (58% over the initial and 57% over the one trained against role-playing). Closer inspection reveals three further patterns: methods for making role-playing LLMs more realistic (e.g., persona conditioning) improve trained assistants but do not close the gap to the fine-tuned simulator; scaling the simulator's model size benefits the fine-tuned simulator but yields no gain for role-playing ones; and assistants trained against role-playing simulators fail to generalize when paired with other simulators at test time, while the one trained against fine-tuned simulator does. Together, these results argue for grounding user simulators in real human behavior and measuring their quality by their downstream effect on real users.

Related papers

• Learning User Simulators with Turing Rewards [32.70383881352914]
  Learning to simulate human users in interactive settings could advance the training of agent assistants, evaluation of personalization systems, and more.<n>We propose Turing-RL: a Turing-Test-based reinforcement learning approach for training user simulator models.
  arXiv  Detail & Related papers  (2026-06-17T17:58:48Z)
• Beyond Cooperative Simulators: Generating Realistic User Personas for Robust Evaluation of LLM Agents [28.85410475505536]
  Persona Policies (PPol) is a control layer that induces realistic behavioral variation in user simulators.<n>PPol produces a diverse population of human-like personas for any task in the domain.<n>Agents trained with PPol are more robust to challenging, out-of-distribution behaviors, improving task success by +17% relative to training only on existing simulated interactions.
  arXiv  Detail & Related papers  (2026-05-13T02:16:51Z)
• Measuring and Mitigating the Distributional Gap Between Real and Simulated User Behaviors [61.610957638373826]
  We introduce a method to measure the distributional gap between real and simulated user behaviors.<n>Given a dataset of real and simulated conversations, our method extracts representations of user behavior from each conversation.<n>We show that most simulators behave similarly, while a few stand apart.
  arXiv  Detail & Related papers  (2026-05-08T15:09:25Z)
• Mind the Sim2Real Gap in User Simulation for Agentic Tasks [101.69142591891234]
  We formalize the Sim2Real gap in user simulation and present the first study running the full $$-bench protocol with real humans.<n> Behaviorally, LLM simulators are excessively cooperative, stylistically uniform, and lack realistic frustration or ambiguity.<n>These findings highlight the importance of human validation when using LLM-based user simulators in the agent development cycle.
  arXiv  Detail & Related papers  (2026-03-11T19:12:31Z)
• HumanLM: Simulating Users with State Alignment Beats Response Imitation [84.89761487596844]
  We propose a novel training framework, HumanLM, which builds user simulators that accurately reflect real users.<n>HumanLM generates natural-language latent states that align with ground-truth responses through reinforcement learning.<n>For extensive evaluation, we develop Humanual, a comprehensive benchmark for simulating real users based on public data.
  arXiv  Detail & Related papers  (2026-02-07T20:26:28Z)
• Consistently Simulating Human Personas with Multi-Turn Reinforcement Learning [52.07170679746533]
  Large Language Models (LLMs) are increasingly used to simulate human users in interactive settings such as therapy, education, and social role-play.<n>We introduce a unified framework for evaluating and improving persona consistency in LLM-generated dialogue.<n>We define three automatic metrics: prompt-to-line consistency, line-to-line consistency, and Q&A consistency, that capture different types of persona drift and validate each against human annotations.
  arXiv  Detail & Related papers  (2025-10-31T19:40:41Z)
• Flipping the Dialogue: Training and Evaluating User Language Models [31.119620506835677]
  We introduce purpose-built User Language Models (User LMs)<n>User LMs are models post-trained to simulate human users in multi-turn conversations.<n>We show how User LMs align better with human behavior and achieve better simulation robustness than existing simulation methods.
  arXiv  Detail & Related papers  (2025-10-08T01:04:36Z)
• SimulatorArena: Are User Simulators Reliable Proxies for Multi-Turn Evaluation of AI Assistants? [61.07963107032645]
  Large language models (LLMs) are increasingly used in interactive applications.<n>Human evaluation remains the gold standard for assessing their performance in multi-turn conversations.<n>We introduce SimulatorArena, a benchmark of 909 annotated human-LLM conversations on two interactive tasks.
  arXiv  Detail & Related papers  (2025-10-06T23:17:44Z)
• KAUCUS: Knowledge Augmented User Simulators for Training Language Model Assistants [3.724713116252253]
  An effective instruction-following assistant can be developed by creating a simulator that can generate useful interaction data. Previous user simulators generally lacked diversity, were mostly closed domain, and necessitated rigid schema. We introduce Kaucus, a Knowledge-Augmented User Simulator framework, to outline a process of creating diverse user simulators.
  arXiv  Detail & Related papers  (2024-01-29T06:57:02Z)
• Metaphorical User Simulators for Evaluating Task-oriented Dialogue Systems [80.77917437785773]
  Task-oriented dialogue systems ( TDSs) are assessed mainly in an offline setting or through human evaluation. We propose a metaphorical user simulator for end-to-end TDS evaluation, where we define a simulator to be metaphorical if it simulates user's analogical thinking in interactions with systems. We also propose a tester-based evaluation framework to generate variants, i.e., dialogue systems with different capabilities.
  arXiv  Detail & Related papers  (2022-04-02T05:11:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.