"Ask Me Anything": How Comcast Uses LLMs to Assist Agents in Real Time

• URL: http://arxiv.org/abs/2405.00801v2
• Date: Mon, 6 May 2024 16:15:32 GMT
• Title: "Ask Me Anything": How Comcast Uses LLMs to Assist Agents in Real Time
• Authors: Scott Rome, Tianwen Chen, Raphael Tang, Luwei Zhou, Ferhan Ture,
• Abstract summary: We introduce "Ask Me Anything" (AMA) as an add-on feature to an agent-facing customer service interface. AMA allows agents to ask questions to a large language model (LLM) on demand, as they are handling customer conversations. We find that agents using AMA versus a traditional search experience spend approximately 10% fewer seconds per conversation containing a search.
• Score: 9.497432249460385
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Customer service is how companies interface with their customers. It can contribute heavily towards the overall customer satisfaction. However, high-quality service can become expensive, creating an incentive to make it as cost efficient as possible and prompting most companies to utilize AI-powered assistants, or "chat bots". On the other hand, human-to-human interaction is still desired by customers, especially when it comes to complex scenarios such as disputes and sensitive topics like bill payment. This raises the bar for customer service agents. They need to accurately understand the customer's question or concern, identify a solution that is acceptable yet feasible (and within the company's policy), all while handling multiple conversations at once. In this work, we introduce "Ask Me Anything" (AMA) as an add-on feature to an agent-facing customer service interface. AMA allows agents to ask questions to a large language model (LLM) on demand, as they are handling customer conversations -- the LLM provides accurate responses in real-time, reducing the amount of context switching the agent needs. In our internal experiments, we find that agents using AMA versus a traditional search experience spend approximately 10% fewer seconds per conversation containing a search, translating to millions of dollars of savings annually. Agents that used the AMA feature provided positive feedback nearly 80% of the time, demonstrating its usefulness as an AI-assisted feature for customer care.

Related papers

• Are LLM-based methods good enough for detecting unfair terms of service? [67.49487557224415]
  Large language models (LLMs) are good at parsing long text-based documents. We build a dataset consisting of 12 questions applied individually to a set of privacy policies. Some open-source models are able to provide a higher accuracy compared to some commercial models.
  arXiv  Detail & Related papers  (2024-08-24T09:26:59Z)
• ProductAgent: Benchmarking Conversational Product Search Agent with Asking Clarification Questions [68.81939215223818]
  ProductAgent is a conversational information seeking agent equipped with abilities of strategic clarification question generation and dynamic product retrieval. We develop the agent with strategies for product feature summarization, query generation, and product retrieval. Experiments show that ProductAgent interacts positively with the user and enhances retrieval performance with increasing dialogue turns.
  arXiv  Detail & Related papers  (2024-07-01T03:50:23Z)
• Question Suggestion for Conversational Shopping Assistants Using Product Metadata [24.23400061359442]
  We propose a framework that employs Large Language Models (LLMs) to automatically generate contextual, useful, answerable, fluent and diverse questions about products. Suggesting these questions to customers as helpful suggestions or hints to both start and continue a conversation can result in a smoother and faster shopping experience.
  arXiv  Detail & Related papers  (2024-05-02T21:16:19Z)
• Can AI Assistants Know What They Don't Know? [79.6178700946602]
  An AI assistant's refusal to answer questions it does not know is a crucial method for reducing hallucinations and making the assistant truthful. We construct a model-specific "I don't know" (Idk) dataset for an assistant, which contains its known and unknown questions. After alignment with Idk datasets, the assistant can refuse to answer most its unknown questions.
  arXiv  Detail & Related papers  (2024-01-24T07:34:55Z)
• Salespeople vs SalesBot: Exploring the Role of Educational Value in Conversational Recommender Systems [78.84530426424838]
  Existing conversational recommender systems often overlook users' lack of background knowledge, focusing solely on gathering preferences. We introduce SalesOps, a framework that facilitates the simulation and evaluation of such systems. We build SalesBot and ShopperBot, a pair of LLM-powered agents that can simulate either side of the framework.
  arXiv  Detail & Related papers  (2023-10-26T19:44:06Z)
• A system for Human-AI collaboration for Online Customer Support [16.22226476879187]
  We present a system where a human support agent collaborates in real-time with an AI agent to satisfactorily answer customer queries. We describe the user interaction elements of the solution, along with the machine learning techniques involved in the AI agent.
  arXiv  Detail & Related papers  (2023-01-28T11:07:23Z)
• INSCIT: Information-Seeking Conversations with Mixed-Initiative Interactions [47.90088587508672]
  InSCIt is a dataset for Information-Seeking Conversations with mixed-initiative Interactions. It contains 4.7K user-agent turns from 805 human-human conversations. We report results of two systems based on state-of-the-art models of conversational knowledge identification and open-domain question answering.
  arXiv  Detail & Related papers  (2022-07-02T06:18:12Z)
• TWEETSUMM -- A Dialog Summarization Dataset for Customer Service [13.661851509322455]
  We introduce the first large scale, high quality, customer care dialog summarization dataset with close to 6500 human annotated summaries. The data is based on real-world customer support dialogs and includes both extractive and abstractive summaries. We also introduce a new unsupervised, extractive summarization method specific to dialogs.
  arXiv  Detail & Related papers  (2021-11-23T14:13:51Z)
• Unsupervised Contextual Paraphrase Generation using Lexical Control and Reinforcement Learning [3.2811284938530636]
  We propose an unsupervised frame-work to generate contextual paraphrases using autoregressive models. We also propose an automated metric based on Semantic Similarity, Textual Entailment, Expression Diversity and Fluency to evaluate the quality of contextual paraphrases.
  arXiv  Detail & Related papers  (2021-03-23T18:22:03Z)
• Evaluating Empathetic Chatbots in Customer Service Settings [6.523873187705393]
  We show that a blended skills chatbots model that responds to customer queries is more likely to resemble actual human agent response if it is trained to recognize emotion and exhibit appropriate empathy. For our analysis, we leverage a Twitter customer service dataset containing several million customer->agent dialog examples in customer service contexts from 20 well-known brands.
  arXiv  Detail & Related papers  (2021-01-05T03:34:35Z)
• Can You be More Social? Injecting Politeness and Positivity into Task-Oriented Conversational Agents [60.27066549589362]
  Social language used by human agents is associated with greater users' responsiveness and task completion. The model uses a sequence-to-sequence deep learning architecture, extended with a social language understanding element. Evaluation in terms of content preservation and social language level using both human judgment and automatic linguistic measures shows that the model can generate responses that enable agents to address users' issues in a more socially appropriate way.
  arXiv  Detail & Related papers  (2020-12-29T08:22:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.