Decision-aware User Simulation Agent for Evaluating Conversational Recommender Systems

• URL: http://arxiv.org/abs/2605.05250v1
• Date: Tue, 05 May 2026 15:11:05 GMT
• Title: Decision-aware User Simulation Agent for Evaluating Conversational Recommender Systems
• Abstract summary: Hesitator is a theory-grounded user simulation framework that explicitly models human decision-making under choice overload.<n>Hesitator reproduces established behavioral patterns from psychological economics, demonstrating its ability to model human decision behavior.
• Score: 4.515081840452073
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Conversational recommender systems (CRS) increasingly rely on user simulators for automated evaluation of sales agents. A key requirement for such simulators is the ability to model human decision-making. However, most existing simulation frameworks do not explicitly model the internal decision process, and LLM-based simulators often exhibit unrealistically strong information-processing capabilities, rarely exhibit the hesitation or decision deferral commonly observed in real consumer behavior, resulting in overly high acceptance probabilities. To address this limitation, we propose Hesitator, a theory-grounded user simulation framework that explicitly models human decision-making under choice overload. The framework introduces a modular Decision Module that separates utility-based item selection from overload-aware commitment decisions. Experiments across multiple user simulation frameworks, domains, sales modes, and LLM backbones show that integrating our module consistently mitigates unrealistic behaviors under increasing overload conditions. Furthermore, Hesitator reproduces established behavioral patterns from psychological economics, demonstrating its ability to model human decision behavior.

Related papers

• Simulate, Reason, Decide: Scientific Reasoning with LLMs for Simulation-Driven Decision Making [50.05403024674141]
  MechSim is a mechanism-grounded neuro-symbolic reasoning framework for executable scientific simulators.<n>Our framework represents simulators through a shared structured schema capturing assumptions, variables, mechanism dependencies, and execution traces.<n>We evaluate our approach across multiple high-stakes domains and show that it improves mechanism-level explanation quality, simulator analysis, and downstream decision-making reliability.
  arXiv  Detail & Related papers  (2026-06-03T06:36:51Z)
• SalesSim: Benchmarking and Aligning Multimodal Language Models as Retail User Simulators [63.68151307455963]
  We present SalesSim, a framework and testbed for evaluating the ability of Multimodal Large Language Models (MLLMs) to simulate realistic, persona-driven customer behavior.<n>We measure consistency between the simulator's actions and its persona specifications, as well as conversational quality.<n>Our experiments demonstrate that UserGRPO boosts decision alignment of the baseline model by 13.8% while improving conversational quality.
  arXiv  Detail & Related papers  (2026-05-08T17:59:23Z)
• Mind the Sim2Real Gap in User Simulation for Agentic Tasks [101.69142591891234]
  We formalize the Sim2Real gap in user simulation and present the first study running the full $$-bench protocol with real humans.<n> Behaviorally, LLM simulators are excessively cooperative, stylistically uniform, and lack realistic frustration or ambiguity.<n>These findings highlight the importance of human validation when using LLM-based user simulators in the agent development cycle.
  arXiv  Detail & Related papers  (2026-03-11T19:12:31Z)
• LLM-Based Multi-Agent System for Simulating and Analyzing Marketing and Consumer Behavior [1.6352616029995921]
  We introduce an LLM-powered multi-agent simulation framework that models consumer decisions and social dynamics.<n>In a price-discount marketing scenario, the system delivers actionable strategy-testing outcomes.
  arXiv  Detail & Related papers  (2025-10-20T23:15:44Z)
• PUB: An LLM-Enhanced Personality-Driven User Behaviour Simulator for Recommender System Evaluation [9.841963696576546]
  Personality-driven User Behaviour Simulator (PUB) integrates the Big Five personality traits to model personalised user behaviour.<n>PUB dynamically infers user personality from behavioural logs (e.g., ratings, reviews) and item metadata, then generates synthetic interactions that preserve statistical fidelity to real-world data.<n> Experiments on the Amazon review datasets show that logs generated by PUB closely align with real user behaviour and reveal meaningful associations between personality traits and recommendation outcomes.
  arXiv  Detail & Related papers  (2025-06-05T01:57:36Z)
• Understanding Endogenous Data Drift in Adaptive Models with Recourse-Seeking Users [6.782864450313782]
  We study user strategic behaviors and their interactions with decision-making systems under resource constraints and competitive dynamics.<n>We propose two methods--Fair-top-k and Dynamic Continual Learning--which significantly reduce recourse cost and improve model robustness.<n>Our findings draw connections to economic theories, highlighting how algorithmic decision-making can unintentionally reinforce a higher standard and generate endogenous barriers to entry.
  arXiv  Detail & Related papers  (2025-03-12T12:17:34Z)
• LLM-Powered User Simulator for Recommender System [29.328839982869923]
  We introduce an LLM-powered user simulator to simulate user engagement with items in an explicit manner.<n>Specifically, we identify the explicit logic of user preferences, leverage LLMs to analyze item characteristics and distill user sentiments.<n>We propose an ensemble model that synergizes logical and statistical insights for user interaction simulations.
  arXiv  Detail & Related papers  (2024-12-22T12:00:04Z)
• LLM experiments with simulation: Large Language Model Multi-Agent System for Simulation Model Parametrization in Digital Twins [4.773175285216063]
  This paper presents a novel framework that applies large language models (LLMs) to automate the parametrization of simulation models in digital twins. The proposed approach enhances the usability of simulation model by infusing it with knowledges from LLM. The system has the potential to increase user-friendliness and reduce the cognitive load on human users.
  arXiv  Detail & Related papers  (2024-05-28T11:59:40Z)
• Modeling Boundedly Rational Agents with Latent Inference Budgets [56.24971011281947]
  We introduce a latent inference budget model (L-IBM) that models agents' computational constraints explicitly. L-IBMs make it possible to learn agent models using data from diverse populations of suboptimal actors. We show that L-IBMs match or outperform Boltzmann models of decision-making under uncertainty.
  arXiv  Detail & Related papers  (2023-12-07T03:55:51Z)
• On Generative Agents in Recommendation [58.42840923200071]
  Agent4Rec is a user simulator in recommendation based on Large Language Models. Each agent interacts with personalized recommender models in a page-by-page manner.
  arXiv  Detail & Related papers  (2023-10-16T06:41:16Z)
• User Behavior Simulation with Large Language Model based Agents [116.74368915420065]
  We propose an LLM-based agent framework and design a sandbox environment to simulate real user behaviors. Based on extensive experiments, we find that the simulated behaviors of our method are very close to the ones of real humans.
  arXiv  Detail & Related papers  (2023-06-05T02:58:35Z)
• DeforestVis: Behavior Analysis of Machine Learning Models with Surrogate Decision Stumps [46.58231605323107]
  We propose DeforestVis, a visual analytics tool that offers summarization of the behaviour of complex ML models. DeforestVis helps users to explore the complexity versus fidelity trade-off by incrementally generating more stumps. We show the applicability and usefulness of DeforestVis with two use cases and expert interviews with data analysts and model developers.
  arXiv  Detail & Related papers  (2023-03-31T21:17:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.