MUSE: Multi-Domain Chinese User Simulation via Self-Evolving Profiles and Rubric-Guided Alignment

• URL: http://arxiv.org/abs/2604.13828v1
• Date: Wed, 15 Apr 2026 13:01:00 GMT
• Title: MUSE: Multi-Domain Chinese User Simulation via Self-Evolving Profiles and Rubric-Guided Alignment
• Abstract summary: MUSE is a Chinese user simulation framework designed to generate human-like, controllable, and behaviorally consistent responses.<n>We propose Iterative Profile Self-Evolution and Role-Reversal Supervised Fine-Tuning to improve local response realism and human-like expression.<n>Experiments show that MUSE consistently outperforms strong baselines in both utterance-level and session-level evaluations.
• Score: 15.586402133245313
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: User simulators are essential for the scalable training and evaluation of interactive AI systems. However, existing approaches often rely on shallow user profiling, struggle to maintain persona consistency over long interactions, and are largely limited to English or single-domain settings. We present MUSE, a multi-domain Chinese user simulation framework designed to generate human-like, controllable, and behaviorally consistent responses. First, we propose Iterative Profile Self-Evolution (IPSE), which gradually optimizes user profiles by comparing and reasoning discrepancies between simulated trajectories and real dialogue behaviors. We then apply Role-Reversal Supervised Fine-Tuning to improve local response realism and human-like expression. To enable fine-grained behavioral alignment, we further train a specialized rubric-based reward model and incorporate it into rubric-guided multi-turn reinforcement learning, which optimizes the simulator at the dialogue level and enhances long-horizon behavioral consistency. Experiments show that MUSE consistently outperforms strong baselines in both utterance-level and session-level evaluations, generating responses that are more realistic, coherent, and persona-consistent over extended interactions.

Related papers

• From Static Context to Calibrated Interactive RL: Mitigating Distribution Shift in Multi-turn Dialogue with Aligned Simulator [33.62561779855603]
  A long-standing goal of the research community is to develop highly interactive LLM-based dialogue agents.<n>Recent research focuses on optimizing policies based on fixed offline logs (Static Context RL) or using a prompt-based simulator (Interactive RL)<n>We show that both paradigms are fundamentally limited by context distribution shift.<n>We propose Calibrated Interactive RL, a unified framework that couples interactive RL with simulator alignment.
  arXiv  Detail & Related papers  (2026-05-26T00:25:12Z)
• Beyond Cooperative Simulators: Generating Realistic User Personas for Robust Evaluation of LLM Agents [28.85410475505536]
  Persona Policies (PPol) is a control layer that induces realistic behavioral variation in user simulators.<n>PPol produces a diverse population of human-like personas for any task in the domain.<n>Agents trained with PPol are more robust to challenging, out-of-distribution behaviors, improving task success by +17% relative to training only on existing simulated interactions.
  arXiv  Detail & Related papers  (2026-05-13T02:16:51Z)
• Measuring and Mitigating the Distributional Gap Between Real and Simulated User Behaviors [61.610957638373826]
  We introduce a method to measure the distributional gap between real and simulated user behaviors.<n>Given a dataset of real and simulated conversations, our method extracts representations of user behavior from each conversation.<n>We show that most simulators behave similarly, while a few stand apart.
  arXiv  Detail & Related papers  (2026-05-08T15:09:25Z)
• Towards Real-world Human Behavior Simulation: Benchmarking Large Language Models on Long-horizon, Cross-scenario, Heterogeneous Behavior Traces [81.41397370235102]
  We introduce OmniBehavior, the first user simulation benchmark constructed entirely from real-world data.<n>We show that current models struggle to accurately simulate complex behaviors, with performance plateauing even as context windows expand.<n>This results in the loss of individual differences and long-tail behaviors, highlighting critical directions for future high-fidelity simulation research.
  arXiv  Detail & Related papers  (2026-04-09T15:26:21Z)
• Mind the Sim2Real Gap in User Simulation for Agentic Tasks [101.69142591891234]
  We formalize the Sim2Real gap in user simulation and present the first study running the full $$-bench protocol with real humans.<n> Behaviorally, LLM simulators are excessively cooperative, stylistically uniform, and lack realistic frustration or ambiguity.<n>These findings highlight the importance of human validation when using LLM-based user simulators in the agent development cycle.
  arXiv  Detail & Related papers  (2026-03-11T19:12:31Z)
• HumanLM: Simulating Users with State Alignment Beats Response Imitation [84.89761487596844]
  We propose a novel training framework, HumanLM, which builds user simulators that accurately reflect real users.<n>HumanLM generates natural-language latent states that align with ground-truth responses through reinforcement learning.<n>For extensive evaluation, we develop Humanual, a comprehensive benchmark for simulating real users based on public data.
  arXiv  Detail & Related papers  (2026-02-07T20:26:28Z)
• Beyond Offline A/B Testing: Context-Aware Agent Simulation for Recommender System Evaluation [0.7031557790463293]
  We introduce ContextSim, an agent framework that simulates believable user proxies by anchoring interactions in daily life activities.<n>To align preferences with genuine humans, we model agents' internal thoughts and enforce consistency at both the action and trajectory levels.
  arXiv  Detail & Related papers  (2026-01-26T05:01:00Z)
• UserLM-R1: Modeling Human Reasoning in User Language Models with Multi-Reward Reinforcement Learning [32.51053667574764]
  We propose UserLM-R1, a novel user language model with reasoning capability.<n>We first construct comprehensive user profiles with both static roles and dynamic scenario-specific goals for adaptation to diverse scenarios.<n>Then, we propose a goal-driven decision-making policy to generate high-quality rationales before producing responses.
  arXiv  Detail & Related papers  (2026-01-14T06:42:01Z)
• Consistently Simulating Human Personas with Multi-Turn Reinforcement Learning [52.07170679746533]
  Large Language Models (LLMs) are increasingly used to simulate human users in interactive settings such as therapy, education, and social role-play.<n>We introduce a unified framework for evaluating and improving persona consistency in LLM-generated dialogue.<n>We define three automatic metrics: prompt-to-line consistency, line-to-line consistency, and Q&A consistency, that capture different types of persona drift and validate each against human annotations.
  arXiv  Detail & Related papers  (2025-10-31T19:40:41Z)
• Sample-Efficient Online Learning in LM Agents via Hindsight Trajectory Rewriting [92.57796055887995]
  We introduce ECHO, a prompting framework that adapts hindsight experience replay from reinforcement learning for language model agents.<n> ECHO generates optimized trajectories for alternative goals that could have been achieved during failed attempts.<n>We evaluate ECHO on stateful versions of XMiniGrid, a text-based navigation and planning benchmark, and PeopleJoinQA, a collaborative information-gathering enterprise simulation.
  arXiv  Detail & Related papers  (2025-10-11T18:11:09Z)
• Know You First and Be You Better: Modeling Human-Like User Simulators via Implicit Profiles [37.43150003866563]
  We introduce the User Simulator with Implicit Profiles (USP), a framework that infers implicit user profiles from human-machine interactions to simulate personalized and realistic dialogues.<n>USP outperforms strong baselines in terms of authenticity and diversity while maintaining comparable consistency.
  arXiv  Detail & Related papers  (2025-02-26T09:26:54Z)
• User Behavior Simulation with Large Language Model based Agents [116.74368915420065]
  We propose an LLM-based agent framework and design a sandbox environment to simulate real user behaviors. Based on extensive experiments, we find that the simulated behaviors of our method are very close to the ones of real humans.
  arXiv  Detail & Related papers  (2023-06-05T02:58:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.