Salespeople vs SalesBot: Exploring the Role of Educational Value in Conversational Recommender Systems

• URL: http://arxiv.org/abs/2310.17749v1
• Date: Thu, 26 Oct 2023 19:44:06 GMT
• Title: Salespeople vs SalesBot: Exploring the Role of Educational Value in Conversational Recommender Systems
• Authors: Lidiya Murakhovs'ka, Philippe Laban, Tian Xie, Caiming Xiong, Chien-Sheng Wu
• Abstract summary: Existing conversational recommender systems often overlook users' lack of background knowledge, focusing solely on gathering preferences. We introduce SalesOps, a framework that facilitates the simulation and evaluation of such systems. We build SalesBot and ShopperBot, a pair of LLM-powered agents that can simulate either side of the framework.
• Score: 78.84530426424838
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Making big purchases requires consumers to research or consult a salesperson to gain domain expertise. However, existing conversational recommender systems (CRS) often overlook users' lack of background knowledge, focusing solely on gathering preferences. In this work, we define a new problem space for conversational agents that aim to provide both product recommendations and educational value through mixed-type mixed-initiative dialog. We introduce SalesOps, a framework that facilitates the simulation and evaluation of such systems by leveraging recent advancements in large language models (LLMs). We build SalesBot and ShopperBot, a pair of LLM-powered agents that can simulate either side of the framework. A comprehensive human study compares SalesBot against professional salespeople, revealing that although SalesBot approaches professional performance in terms of fluency and informativeness, it lags behind in recommendation quality. We emphasize the distinct limitations both face in providing truthful information, highlighting the challenges of ensuring faithfulness in the CRS context. We release our code and make all data available.

Related papers

• Federated Learning for Data Market: Shapley-UCB for Seller Selection and Incentives [0.3069335774032178]
  We propose a transaction framework based on the federated learning architecture, and design a seller selection algorithm and incentive compensation mechanism. Specifically, we use gradient similarity and Shapley algorithm to fairly and accurately evaluate the contribution of sellers. After the training, fair compensation is made according to the seller's participation in the training.
  arXiv  Detail & Related papers  (2024-10-10T03:50:20Z)
• Injecting Salesperson's Dialogue Strategies in Large Language Models with Chain-of-Thought Reasoning [23.919423630938226]
  SalesBot simulates dialogues transitioning from chit-chat to task-oriented scenarios to train sales agents. Initial data lacked smooth transitions and coherent long-turn dialogues, resulting in poor naturalness in sales-customer interactions. We introduce a novel model called SalesAgent, trained on salesperson's interactions, using chain-of-thought (CoT) reasoning.
  arXiv  Detail & Related papers  (2024-04-29T10:12:04Z)
• CAUSE: Counterfactual Assessment of User Satisfaction Estimation in Task-Oriented Dialogue Systems [60.27663010453209]
  We leverage large language models (LLMs) to generate satisfaction-aware counterfactual dialogues. We gather human annotations to ensure the reliability of the generated samples. Our results shed light on the need for data augmentation approaches for user satisfaction estimation in TOD systems.
  arXiv  Detail & Related papers  (2024-03-27T23:45:31Z)
• Measuring Bargaining Abilities of LLMs: A Benchmark and A Buyer-Enhancement Method [17.388837360641276]
  This paper describes the Bargaining task as an asymmetric incomplete information game. It allows us to quantitatively assess an agent's performance in the Bargain task. We propose a novel approach called OG-Narrator that integrates a deterministic Offer Generator to control the price range of Buyer's offers.
  arXiv  Detail & Related papers  (2024-02-24T13:36:58Z)
• Conversational Recommender System and Large Language Model Are Made for Each Other in E-commerce Pre-sales Dialogue [80.51690477289418]
  Conversational recommender systems (CRSs) learn user representation and provide accurate recommendations based on dialogue context, but rely on external knowledge. Large language models (LLMs) generate responses that mimic pre-sales dialogues after fine-tuning, but lack domain-specific knowledge for accurate recommendations. This paper investigates the effectiveness of combining LLM and CRS in E-commerce pre-sales dialogues.
  arXiv  Detail & Related papers  (2023-10-23T07:00:51Z)
• UltraFeedback: Boosting Language Models with Scaled AI Feedback [99.4633351133207]
  We present textscUltraFeedback, a large-scale, high-quality, and diversified AI feedback dataset. Our work validates the effectiveness of scaled AI feedback data in constructing strong open-source chat language models.
  arXiv  Detail & Related papers  (2023-10-02T17:40:01Z)
• Rethinking the Evaluation for Conversational Recommendation in the Era of Large Language Models [115.7508325840751]
  The recent success of large language models (LLMs) has shown great potential to develop more powerful conversational recommender systems (CRSs) In this paper, we embark on an investigation into the utilization of ChatGPT for conversational recommendation, revealing the inadequacy of the existing evaluation protocol. We propose an interactive Evaluation approach based on LLMs named iEvaLM that harnesses LLM-based user simulators.
  arXiv  Detail & Related papers  (2023-05-22T15:12:43Z)
• Approximating Online Human Evaluation of Social Chatbots with Prompting [11.657633779338724]
  Existing evaluation metrics aim to automate offline user evaluation and approximate human judgment of pre-curated dialogs. We propose an approach to approximate online human evaluation leveraging large language models (LLMs) from the GPT family. We introduce a new Dialog system Evaluation framework based on Prompting (DEP), which enables a fully automatic evaluation pipeline.
  arXiv  Detail & Related papers  (2023-04-11T14:45:01Z)
• Intent Recognition in Conversational Recommender Systems [0.0]
  We introduce a pipeline to contextualize the input utterances in conversations. We then take the next step towards leveraging reverse feature engineering to link the contextualized input and learning model to support intent recognition.
  arXiv  Detail & Related papers  (2022-12-06T11:02:42Z)
• Seamlessly Unifying Attributes and Items: Conversational Recommendation for Cold-Start Users [111.28351584726092]
  We consider the conversational recommendation for cold-start users, where a system can both ask the attributes from and recommend items to a user interactively. Our Conversational Thompson Sampling (ConTS) model holistically solves all questions in conversational recommendation by choosing the arm with the maximal reward to play.
  arXiv  Detail & Related papers  (2020-05-23T08:56:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.