SalesSim: Benchmarking and Aligning Multimodal Language Models as Retail User Simulators

• URL: http://arxiv.org/abs/2605.08334v1
• Date: Fri, 08 May 2026 17:59:23 GMT
• Title: SalesSim: Benchmarking and Aligning Multimodal Language Models as Retail User Simulators
• Abstract summary: We present SalesSim, a framework and testbed for evaluating the ability of Multimodal Large Language Models (MLLMs) to simulate realistic, persona-driven customer behavior.<n>We measure consistency between the simulator's actions and its persona specifications, as well as conversational quality.<n>Our experiments demonstrate that UserGRPO boosts decision alignment of the baseline model by 13.8% while improving conversational quality.
• Score: 63.68151307455963
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present SalesSim, a framework and testbed for evaluating the ability of Multimodal Large Language Models (MLLMs) to simulate realistic, persona-driven customer behavior in multi-turn, multi-modal, tool-augmented online retail conversations. Unlike prior work that treat user simulation as surface-level dialogue generation, SalesSim models retail interaction and decision-making as a grounded, agentic process, where shoppers with diverse backgrounds, preferences, and dealbreakers interact with a sales agent, seek clarifications, and make informed purchasing decisions. For evaluation, we design a suite of metrics centered on decision alignment, measuring the consistency between the simulator's actions and its persona specifications, as well as conversational quality. We find several behavioral gaps after benchmarking 6 open and closed-source state-of-the-art models. First, while models produce fluent conversations, they display significantly lower lexical diversity and overdisclosure of criteria across personas compared to human conversations. Second, models tend to be persuaded by sales agent suggestions and drift from persona specifications. Even the strongest model achieves less than 79% average alignment with its underlying persona specifications. To make progress on these limitations, we propose UserGRPO, a multi-turn, multi-objective reinforcement learning recipe to optimize both conversational fluency and decision alignment under persona specifications. Our experiments demonstrate that UserGRPO boosts decision alignment of the baseline model by 13.8% while improving conversational quality. By introducing SalesSim, we provide a new testbed for the community to investigate and improve the adherence of user simulators in goal-oriented settings.

Related papers

• Decision-aware User Simulation Agent for Evaluating Conversational Recommender Systems [4.515081840452073]
  Hesitator is a theory-grounded user simulation framework that explicitly models human decision-making under choice overload.<n>Hesitator reproduces established behavioral patterns from psychological economics, demonstrating its ability to model human decision behavior.
  arXiv  Detail & Related papers  (2026-05-05T15:11:05Z)
• Sell More, Play Less: Benchmarking LLM Realistic Selling Skill [1.1560747347923173]
  SalesLLM benchmark is built from realistic applications covering Financial Services and Consumer Goods.<n>SalesLLM benchmark scores correlate strongly with expert human ratings.
  arXiv  Detail & Related papers  (2026-04-08T13:06:37Z)
• What Makes a Sale? Rethinking End-to-End Seller--Buyer Retail Dynamics with LLM Agents [22.408631930364788]
  We present RetailSim, an end-to-end retail simulation framework that models this pipeline in a unified environment.<n>We demonstrate its practical utility via decision-oriented use cases, including persona inference, seller-buyer interaction analysis, and sales strategy evaluation.
  arXiv  Detail & Related papers  (2026-04-06T06:38:35Z)
• ShopSimulator: Evaluating and Exploring RL-Driven LLM Agent for Shopping Assistants [30.950450014358722]
  We introduce ShopSimulator, a large-scale and challenging Chinese shopping environment.<n>We evaluate LLMs across diverse scenarios, finding that even the best-performing models achieve less than 40% full-success rate.<n>Further training exploration provides practical guidance for overcoming these weaknesses.
  arXiv  Detail & Related papers  (2026-01-26T07:24:28Z)
• A Personalized Conversational Benchmark: Towards Simulating Personalized Conversations [112.81207927088117]
  PersonaConvBench is a benchmark for evaluating personalized reasoning and generation in multi-turn conversations with large language models (LLMs)<n>We benchmark several commercial and open-source LLMs under a unified prompting setup and observe that incorporating personalized history yields substantial performance improvements.
  arXiv  Detail & Related papers  (2025-05-20T09:13:22Z)
• VisualPRM: An Effective Process Reward Model for Multimodal Reasoning [76.35753243272521]
  We introduce VisualPRM, which improves the reasoning abilities of existing Multimodal Large Language Models (MLLMs)<n>Our model achieves a 5.9-point improvement across seven multimodal reasoning benchmarks.<n>For the evaluation of multimodal PRMs, we propose VisualProcessBench, a benchmark with human-annotated step-wise correctness labels.
  arXiv  Detail & Related papers  (2025-03-13T12:03:37Z)
• Salespeople vs SalesBot: Exploring the Role of Educational Value in Conversational Recommender Systems [78.84530426424838]
  Existing conversational recommender systems often overlook users' lack of background knowledge, focusing solely on gathering preferences. We introduce SalesOps, a framework that facilitates the simulation and evaluation of such systems. We build SalesBot and ShopperBot, a pair of LLM-powered agents that can simulate either side of the framework.
  arXiv  Detail & Related papers  (2023-10-26T19:44:06Z)
• Generative Judge for Evaluating Alignment [84.09815387884753]
  We propose a generative judge with 13B parameters, Auto-J, designed to address these challenges. Our model is trained on user queries and LLM-generated responses under massive real-world scenarios. Experimentally, Auto-J outperforms a series of strong competitors, including both open-source and closed-source models.
  arXiv  Detail & Related papers  (2023-10-09T07:27:15Z)
• Unlocking the Potential of User Feedback: Leveraging Large Language Model as User Simulator to Enhance Dialogue System [65.93577256431125]
  We propose an alternative approach called User-Guided Response Optimization (UGRO) to combine it with a smaller task-oriented dialogue model. This approach uses LLM as annotation-free user simulator to assess dialogue responses, combining them with smaller fine-tuned end-to-end TOD models. Our approach outperforms previous state-of-the-art (SOTA) results.
  arXiv  Detail & Related papers  (2023-06-16T13:04:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.