Customer Sentiment Analysis using Weak Supervision for Customer-Agent Chat

• URL: http://arxiv.org/abs/2111.14282v2
• Date: Tue, 30 Nov 2021 03:38:39 GMT
• Title: Customer Sentiment Analysis using Weak Supervision for Customer-Agent Chat
• Authors: Navdeep Jain
• Abstract summary: We perform sentiment analysis on customer chat using weak supervision on our in-house dataset. We fine-tune the pre-trained language model (LM) RoBERTa as a sentiment classifier using weak supervision.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Prior work on sentiment analysis using weak supervision primarily focuses on different reviews such as movies (IMDB), restaurants (Yelp), products (Amazon).~One under-explored field in this regard is customer chat data for a customer-agent chat in customer support due to the lack of availability of free public data. Here, we perform sentiment analysis on customer chat using weak supervision on our in-house dataset. We fine-tune the pre-trained language model (LM) RoBERTa as a sentiment classifier using weak supervision. Our contribution is as follows:1) We show that by using weak sentiment classifiers along with domain-specific lexicon-based rules as Labeling Functions (LF), we can train a fairly accurate customer chat sentiment classifier using weak supervision. 2) We compare the performance of our custom-trained model with off-the-shelf google cloud NLP API for sentiment analysis. We show that by injecting domain-specific knowledge using LFs, even with weak supervision, we can train a model to handle some domain-specific use cases better than off-the-shelf google cloud NLP API. 3) We also present an analysis of how customer sentiment in a chat relates to problem resolution.

Related papers

• Using LLMs to Establish Implicit User Sentiment of Software Desirability [0.0]
  This study explores the use of LLMs for providing quantitative zero-shot sentiment analysis of implicit software desirability. Data is collected using the Microsoft Product Desirability Toolkit (PDT), a well-known qualitative user experience analysis tool.
  arXiv  Detail & Related papers  (2024-08-02T18:40:10Z)
• Unsupervised Sentiment Analysis of Plastic Surgery Social Media Posts [91.3755431537592]
  The massive collection of user posts across social media platforms is primarily untapped for artificial intelligence (AI) use cases. Natural language processing (NLP) is a subfield of AI that leverages bodies of documents, known as corpora, to train computers in human-like language understanding. This study demonstrates that the applied results of unsupervised analysis allow a computer to predict either negative, positive, or neutral user sentiment towards plastic surgery.
  arXiv  Detail & Related papers  (2023-07-05T20:16:20Z)
• Bring Your Own Data! Self-Supervised Evaluation for Large Language Models [52.15056231665816]
  We propose a framework for self-supervised evaluation of Large Language Models (LLMs) We demonstrate self-supervised evaluation strategies for measuring closed-book knowledge, toxicity, and long-range context dependence. We find strong correlations between self-supervised and human-supervised evaluations.
  arXiv  Detail & Related papers  (2023-06-23T17:59:09Z)
• Leveraging ChatGPT As Text Annotation Tool For Sentiment Analysis [6.596002578395151]
  ChatGPT is a new product of OpenAI and has emerged as the most popular AI product. This study explores the use of ChatGPT as a tool for data labeling for different sentiment analysis tasks.
  arXiv  Detail & Related papers  (2023-06-18T12:20:42Z)
• REaaS: Enabling Adversarially Robust Downstream Classifiers via Robust Encoder as a Service [67.0982378001551]
  We show how a service provider pre-trains an encoder and then deploys it as a cloud service API. A client queries the cloud service API to obtain feature vectors for its training/testing inputs. We show that the cloud service only needs to provide two APIs to enable a client to certify the robustness of its downstream classifier.
  arXiv  Detail & Related papers  (2023-01-07T17:40:11Z)
• Proactive Detractor Detection Framework Based on Message-Wise Sentiment Analysis Over Customer Support Interactions [60.87845704495664]
  We propose a framework relying solely on chat-based customer support interactions for predicting the recommendation decision of individual users. For our case study, we analyzed a total number of 16.4k users and 48.7k customer support conversations within the financial vertical of a large e-commerce company in Latin America. Our results show that, with respective feature interpretability, it is possible to predict the likelihood of a user to recommend a product or service, based solely on the message-wise sentiment evolution of their CS conversations in a fully automated way.
  arXiv  Detail & Related papers  (2022-11-08T00:43:36Z)
• Efficient Few-Shot Fine-Tuning for Opinion Summarization [83.76460801568092]
  Abstractive summarization models are typically pre-trained on large amounts of generic texts, then fine-tuned on tens or hundreds of thousands of annotated samples. We show that a few-shot method based on adapters can easily store in-domain knowledge. We show that this self-supervised adapter pre-training improves summary quality over standard fine-tuning by 2.0 and 1.3 ROUGE-L points on the Amazon and Yelp datasets.
  arXiv  Detail & Related papers  (2022-05-04T16:38:37Z)
• What is wrong with you?: Leveraging User Sentiment for Automatic Dialog Evaluation [73.03318027164605]
  We propose to use information that can be automatically extracted from the next user utterance as a proxy to measure the quality of the previous system response. Our model generalizes across both spoken and written open-domain dialog corpora collected from real and paid users.
  arXiv  Detail & Related papers  (2022-03-25T22:09:52Z)
• Data Driven Content Creation using Statistical and Natural Language Processing Techniques for Financial Domain [0.0]
  We propose a two-part framework where the first part describes methods to combine the information from different interaction channels like call, search, and chat. The second part of the framework focuses on extracting customer questions by analyzing interaction data sources.
  arXiv  Detail & Related papers  (2021-09-07T08:37:28Z)
• Evaluating Empathetic Chatbots in Customer Service Settings [6.523873187705393]
  We show that a blended skills chatbots model that responds to customer queries is more likely to resemble actual human agent response if it is trained to recognize emotion and exhibit appropriate empathy. For our analysis, we leverage a Twitter customer service dataset containing several million customer->agent dialog examples in customer service contexts from 20 well-known brands.
  arXiv  Detail & Related papers  (2021-01-05T03:34:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.