Controllable User Simulation

• URL: http://arxiv.org/abs/2605.11519v1
• Date: Tue, 12 May 2026 04:44:29 GMT
• Title: Controllable User Simulation
• Abstract summary: This work formalizes controllable simulation as a causal inference problem.<n>We show that the standard practice of training simulators via supervised fine-tuning on post-hoc trajectory labels yields a structurally biased model.<n>We propose practical training mitigations: a priori controls, step-wise dynamic controls, and direct policy-conditioned learning.
• Score: 41.65602738642653
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Using offline datasets to evaluate conversational agents often fails to cover rare scenarios or to support testing new policies. This has motivated the use of controllable user simulators for targeted, counterfactual evaluation, typically implemented by prompting or fine-tuning large language models. In this work, we formalize controllable simulation as a causal inference problem. By bridging natural language evaluation with off-policy evaluation methodology, we show that the standard practice of training simulators via supervised fine-tuning on post-hoc trajectory labels yields a structurally biased model. Specifically, these labels are inextricably coupled to the data-generating behavior policy, injecting a look-ahead bias that breaks causal consistency. Furthermore, we prove that under policy shift this failure causes the variance of evaluation metrics to explode geometrically, a phenomenon we term controllability collapse. To restore causal consistency, we establish theoretical conditions for accurate simulation and propose practical training mitigations: a priori controls, step-wise dynamic controls, and direct policy-conditioned learning. Empirical evaluation confirms that while standard global controls distort conversational distributions and collapse behavioral diversity, our causally grounded simulators eliminate look-ahead bias, preserve natural variance, and exhibit robust zero-shot generalization to unseen agent behaviors.

Related papers

• WorldSimProbe: Diagnosing Simulator Faithfulness in Action-Conditioned World Models for Embodied Manipulation [85.15806404962193]
  Action-conditioned world models (ACWMs) promise to provide embodied AI with scalable predictive simulators for planning, policy evaluation, and data generation.<n>We evaluate ACWMs through the observable capabilities expected of physical simulators.<n>WorldSimProbe reveals systematic action-realization degradation across control variation, structured failures in interaction grounding and dynamics.
  arXiv  Detail & Related papers  (2026-08-10T08:48:06Z)
• Do LLM-Driven Agents Exhibit Engagement Mechanisms? Controlled Tests of Information Load, Descriptive Norms, and Popularity Cues [28.621781661498545]
  We evaluate what an LLM-driven simulation can credibly support using information engagement on social media as a test case.<n>In a Weibo-like environment, we manipulate information load and descriptive norms, while allowing popularity cues to evolve endogenously.<n>We ask whether simulated behavior changes in theoretically interpretable ways under these controlled variations, rather than merely producing plausible-looking traces.
  arXiv  Detail & Related papers  (2026-03-21T18:50:22Z)
• Sim2Act: Robust Simulation-to-Decision Learning via Adversarial Calibration and Group-Relative Perturbation [54.29523408543184]
  Simulation-to-decision learning enables safe policy training in digital environments without risking real-world deployment.<n>Existing approaches either focus on improving average simulation fidelity or adopt conservative regularization.<n>We propose Sim2Act, a robust simulation-to-decision framework that addresses both simulator and policy robustness.
  arXiv  Detail & Related papers  (2026-03-10T00:51:47Z)
• Drift-Bench: Diagnosing Cooperative Breakdowns in LLM Agents under Input Faults via Multi-Turn Interaction [20.610305266852638]
  textbfDrift-Bench is the first diagnostic benchmark that evaluates agentic pragmatics under input faults.<n>Method bridges clarification research and agent safety evaluation, enabling systematic diagnosis of failures that can lead to unsafe executions.
  arXiv  Detail & Related papers  (2026-02-02T18:46:16Z)
• Causal Imitation Learning Under Measurement Error and Distribution Shift [6.038778620145853]
  We study offline imitation learning (IL) when part of the decision-relevant state is observed only through noisy measurements.<n>We propose a general framework for IL under measurement error, inspired by explicitly modeling the causal relationships among the variables.
  arXiv  Detail & Related papers  (2026-01-29T18:06:53Z)
• Active Inference with Reusable State-Dependent Value Profiles [0.0]
  We introduce value profiles: a small set of reusable bundles of value-related parameters assigned to hidden states in a generative model.<n>We evaluate this framework in probabilistic reversal learning, comparing static-precision, entropy-coupled dynamic-precision, and profile-based models.
  arXiv  Detail & Related papers  (2025-12-03T04:11:57Z)
• Time-series Generation by Contrastive Imitation [87.51882102248395]
  We study a generative framework that seeks to combine the strengths of both: Motivated by a moment-matching objective to mitigate compounding error, we optimize a local (but forward-looking) transition policy. At inference, the learned policy serves as the generator for iterative sampling, and the learned energy serves as a trajectory-level measure for evaluating sample quality.
  arXiv  Detail & Related papers  (2023-11-02T16:45:25Z)
• Bring Your Own Data! Self-Supervised Evaluation for Large Language Models [52.15056231665816]
  We propose a framework for self-supervised evaluation of Large Language Models (LLMs) We demonstrate self-supervised evaluation strategies for measuring closed-book knowledge, toxicity, and long-range context dependence. We find strong correlations between self-supervised and human-supervised evaluations.
  arXiv  Detail & Related papers  (2023-06-23T17:59:09Z)
• Stateful Offline Contextual Policy Evaluation and Learning [88.9134799076718]
  We study off-policy evaluation and learning from sequential data. We formalize the relevant causal structure of problems such as dynamic personalized pricing. We show improved out-of-sample policy performance in this class of relevant problems.
  arXiv  Detail & Related papers  (2021-10-19T16:15:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.