Sell More, Play Less: Benchmarking LLM Realistic Selling Skill

• URL: http://arxiv.org/abs/2604.07054v2
• Date: Thu, 09 Apr 2026 07:49:38 GMT
• Title: Sell More, Play Less: Benchmarking LLM Realistic Selling Skill
• Abstract summary: SalesLLM benchmark is built from realistic applications covering Financial Services and Consumer Goods.<n>SalesLLM benchmark scores correlate strongly with expert human ratings.
• Score: 1.1560747347923173
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Sales dialogues require multi-turn, goal-directed persuasion under asymmetric incentives, which makes them a challenging setting for large language models (LLMs). Yet existing dialogue benchmarks rarely measure deal progression and outcomes. We introduce SalesLLM benchmark, a bilingual (ZH/EN) benchmark derived from realistic applications covering Financial Services and Consumer Goods, built from 30,074 scripted configurations and 1,805 curated multi-turn scenarios with controllable difficulty and personas. We propose a fully automatic evaluation pipeline that combines (i) an LLM-based rater for sales-process progress,and (ii) fine-tuned BERT classifiers for end-of-dialogue buying intent. To improve simulation fidelity, we train a user model, CustomerLM, with SFT and DPO on 8,000+ crowdworker-involved sales conversations, reducing role inversion from 17.44% (GPT-4o) to 8.8%. SalesLLM benchmark scores correlate strongly with expert human ratings (Pearson r=0.98). Experiments across 15 mainstream LLMs reveal substantial variability: top-performance LLMs are competitive with human-level performance while the less capable ones are worse than human. SalesLLM benchmark serves as a scalable benchmark for developing and evaluating outcome-oriented sales agents.

Related papers

• SalesLoop: Reinforcement Learning from Performance Feedback for Sales Lead Ranking [3.337528987886516]
  SalesLoop is a performance-aware reward that encodes conversion outcomes weighted by ranking position and conversion velocity.<n>SalesLoop improves NDCG@K by +7.9% and P@K by +15.8% over the strongest static baseline.
  arXiv  Detail & Related papers  (2026-07-22T18:26:57Z)
• Simulated Customers Never Walk Away: Decision Fidelity of LLM User Simulators Measured Against Real Purchase Outcomes [6.1260812874439345]
  Existing frameworks measure whether simulators talk like humans against ground truth from paid participants.<n>We argue this has a structural blind spot: when goal is assigned, user's motivation is endogenous, latent, and decaying.<n>We introduce decision fidelity -- whether a simulated population reproduces the decision dynamics of real users facing real consequential choices.
  arXiv  Detail & Related papers  (2026-06-16T03:53:10Z)
• Rethinking Sales Lead Scoring with LLM-based Hierarchical Preference Ranking [10.224056739324702]
  Sales lead conversion differs fundamentally from e-commerce recommendation due to prolonged decision cycles and multi-stage funnels.<n>Traditional lead scoring methods face severe challenges: sparse supervision, a semantic gap in unstructured CRM logs, and inability to capture relative lead priority.<n>We introduce an LLM-based discriminative framework for sales lead scoring, which supports joint modeling of structured CRM features and unstructured customer interactions.
  arXiv  Detail & Related papers  (2026-06-03T03:05:57Z)
• SalesSim: Benchmarking and Aligning Multimodal Language Models as Retail User Simulators [63.68151307455963]
  We present SalesSim, a framework and testbed for evaluating the ability of Multimodal Large Language Models (MLLMs) to simulate realistic, persona-driven customer behavior.<n>We measure consistency between the simulator's actions and its persona specifications, as well as conversational quality.<n>Our experiments demonstrate that UserGRPO boosts decision alignment of the baseline model by 13.8% while improving conversational quality.
  arXiv  Detail & Related papers  (2026-05-08T17:59:23Z)
• RankLLM: Weighted Ranking of LLMs by Quantifying Question Difficulty [102.02839046225468]
  RankLLM is a novel framework designed to quantify both question difficulty and model competency.<n>We evaluate 30 models on 35,550 questions across multiple domains.
  arXiv  Detail & Related papers  (2026-02-12T21:28:46Z)
• ShopSimulator: Evaluating and Exploring RL-Driven LLM Agent for Shopping Assistants [30.950450014358722]
  We introduce ShopSimulator, a large-scale and challenging Chinese shopping environment.<n>We evaluate LLMs across diverse scenarios, finding that even the best-performing models achieve less than 40% full-success rate.<n>Further training exploration provides practical guidance for overcoming these weaknesses.
  arXiv  Detail & Related papers  (2026-01-26T07:24:28Z)
• CRMArena-Pro: Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactions [85.88573535033406]
  CRMArena-Pro is a novel benchmark for holistic, realistic assessment of LLM agents in diverse professional settings.<n>It incorporates multi-turn interactions guided by diverse personas and robust confidentiality awareness assessments.<n>Experiments reveal leading LLM agents achieve only around 58% single-turn success on CRMArena-Pro, with performance dropping significantly to approximately 35% in multi-turn settings.
  arXiv  Detail & Related papers  (2025-05-24T21:33:22Z)
• LLM2: Let Large Language Models Harness System 2 Reasoning [65.89293674479907]
  Large language models (LLMs) have exhibited impressive capabilities across a myriad of tasks, yet they occasionally yield undesirable outputs.<n>We introduce LLM2, a novel framework that combines an LLM with a process-based verifier.<n>LLMs2 is responsible for generating plausible candidates, while the verifier provides timely process-based feedback to distinguish desirable and undesirable outputs.
  arXiv  Detail & Related papers  (2024-12-29T06:32:36Z)
• Auto-Arena: Automating LLM Evaluations with Agent Peer Battles and Committee Discussions [77.66677127535222]
  Auto-Arena is an innovative framework that automates the entire evaluation process using LLM-powered agents. In our experiments, Auto-Arena shows a 92.14% correlation with human preferences, surpassing all previous expert-annotated benchmarks.
  arXiv  Detail & Related papers  (2024-05-30T17:19:19Z)
• Measuring Bargaining Abilities of LLMs: A Benchmark and A Buyer-Enhancement Method [17.388837360641276]
  This paper describes the Bargaining task as an asymmetric incomplete information game. It allows us to quantitatively assess an agent's performance in the Bargain task. We propose a novel approach called OG-Narrator that integrates a deterministic Offer Generator to control the price range of Buyer's offers.
  arXiv  Detail & Related papers  (2024-02-24T13:36:58Z)
• LLMs as Factual Reasoners: Insights from Existing Benchmarks and Beyond [135.8013388183257]
  We propose a new protocol for inconsistency detection benchmark creation and implement it in a 10-domain benchmark called SummEdits. Most LLMs struggle on SummEdits, with performance close to random chance. The best-performing model, GPT-4, is still 8% below estimated human performance.
  arXiv  Detail & Related papers  (2023-05-23T21:50:06Z)
• Large Language Models are Not Yet Human-Level Evaluators for Abstractive Summarization [66.08074487429477]
  We investigate the stability and reliability of large language models (LLMs) as automatic evaluators for abstractive summarization. We find that while ChatGPT and GPT-4 outperform the commonly used automatic metrics, they are not ready as human replacements.
  arXiv  Detail & Related papers  (2023-05-22T14:58:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.