Leveraging ChatGPT As Text Annotation Tool For Sentiment Analysis
- URL: http://arxiv.org/abs/2306.17177v1
- Date: Sun, 18 Jun 2023 12:20:42 GMT
- Title: Leveraging ChatGPT As Text Annotation Tool For Sentiment Analysis
- Authors: Mohammad Belal, James She, Simon Wong
- Abstract summary: ChatGPT is a new product of OpenAI and has emerged as the most popular AI product.
This study explores the use of ChatGPT as a tool for data labeling for different sentiment analysis tasks.
- Score: 6.596002578395151
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Sentiment analysis is a well-known natural language processing task that
involves identifying the emotional tone or polarity of a given piece of text.
With the growth of social media and other online platforms, sentiment analysis
has become increasingly crucial for businesses and organizations seeking to
monitor and comprehend customer feedback as well as opinions. Supervised
learning algorithms have been popularly employed for this task, but they
require human-annotated text to create the classifier. To overcome this
challenge, lexicon-based tools have been used. A drawback of lexicon-based
algorithms is their reliance on pre-defined sentiment lexicons, which may not
capture the full range of sentiments in natural language. ChatGPT is a new
product of OpenAI and has emerged as the most popular AI product. It can answer
questions on various topics and tasks. This study explores the use of ChatGPT
as a tool for data labeling for different sentiment analysis tasks. It is
evaluated on two distinct sentiment analysis datasets with varying purposes.
The results demonstrate that ChatGPT outperforms other lexicon-based
unsupervised methods with significant improvements in overall accuracy.
Specifically, compared to the best-performing lexical-based algorithms, ChatGPT
achieves a remarkable increase in accuracy of 20% for the tweets dataset and
approximately 25% for the Amazon reviews dataset. These findings highlight the
exceptional performance of ChatGPT in sentiment analysis tasks, surpassing
existing lexicon-based approaches by a significant margin. The evidence
suggests it can be used for annotation on different sentiment analysis events
and taskss.
Related papers
- You Shall Know a Tool by the Traces it Leaves: The Predictability of Sentiment Analysis Tools [74.98850427240464]
We show that sentiment analysis tools disagree on the same dataset.
We show that the sentiment tool used for sentiment annotation can even be predicted from its outcome.
arXiv Detail & Related papers (2024-10-18T17:27:38Z) - Lexicon-Based Sentiment Analysis on Text Polarities with Evaluation of Classification Models [1.342834401139078]
This work uses a lexicon-based method to perform sentiment analysis and shows an evaluation of classification models trained over textual data.
The lexicon-based methods identify the intensity of emotion and subjectivity at word levels.
This work is based on a multi-class problem of text being labeled as positive, negative, or neutral.
arXiv Detail & Related papers (2024-09-19T15:31:12Z) - SOUL: Towards Sentiment and Opinion Understanding of Language [96.74878032417054]
We propose a new task called Sentiment and Opinion Understanding of Language (SOUL)
SOUL aims to evaluate sentiment understanding through two subtasks: Review (RC) and Justification Generation (JG)
arXiv Detail & Related papers (2023-10-27T06:48:48Z) - Leveraging Explainable AI to Analyze Researchers' Aspect-Based Sentiment
about ChatGPT [3.3340659845424536]
We propose a methodology that uses Explainable AI to facilitate such analysis on research data.
Our technique presents valuable insights into extending the state of the art of Aspect-Based Sentiment Analysis on newer datasets.
arXiv Detail & Related papers (2023-08-16T07:44:06Z) - Playing with Words: Comparing the Vocabulary and Lexical Richness of
ChatGPT and Humans [3.0059120458540383]
generative language models such as ChatGPT have triggered a revolution that can transform how text is generated.
Will the use of tools such as ChatGPT increase or reduce the vocabulary used or the lexical richness?
This has implications for words, as those not included in AI-generated content will tend to be less and less popular and may eventually be lost.
arXiv Detail & Related papers (2023-08-14T21:19:44Z) - TextFormer: A Query-based End-to-End Text Spotter with Mixed Supervision [61.186488081379]
We propose TextFormer, a query-based end-to-end text spotter with Transformer architecture.
TextFormer builds upon an image encoder and a text decoder to learn a joint semantic understanding for multi-task modeling.
It allows for mutual training and optimization of classification, segmentation, and recognition branches, resulting in deeper feature sharing.
arXiv Detail & Related papers (2023-06-06T03:37:41Z) - ChatGraph: Interpretable Text Classification by Converting ChatGPT
Knowledge to Graphs [54.48467003509595]
ChatGPT has shown superior performance in various natural language processing (NLP) tasks.
We propose a novel framework that leverages the power of ChatGPT for specific tasks, such as text classification.
Our method provides a more transparent decision-making process compared with previous text classification methods.
arXiv Detail & Related papers (2023-05-03T19:57:43Z) - Can ChatGPT Reproduce Human-Generated Labels? A Study of Social
Computing Tasks [9.740764281808588]
ChatGPT has the potential to reproduce human-generated label annotations in social computing tasks.
We relabel five datasets covering stance detection (2x), sentiment analysis, hate speech, and bot detection.
Our results highlight that ChatGPT does have the potential to handle these data annotation tasks, although a number of challenges remain.
arXiv Detail & Related papers (2023-04-20T08:08:12Z) - To ChatGPT, or not to ChatGPT: That is the question! [78.407861566006]
This study provides a comprehensive and contemporary assessment of the most recent techniques in ChatGPT detection.
We have curated a benchmark dataset consisting of prompts from ChatGPT and humans, including diverse questions from medical, open Q&A, and finance domains.
Our evaluation results demonstrate that none of the existing methods can effectively detect ChatGPT-generated content.
arXiv Detail & Related papers (2023-04-04T03:04:28Z) - Weakly-Supervised Aspect-Based Sentiment Analysis via Joint
Aspect-Sentiment Topic Embedding [71.2260967797055]
We propose a weakly-supervised approach for aspect-based sentiment analysis.
We learn sentiment, aspect> joint topic embeddings in the word embedding space.
We then use neural models to generalize the word-level discriminative information.
arXiv Detail & Related papers (2020-10-13T21:33:24Z) - Tweets Sentiment Analysis via Word Embeddings and Machine Learning
Techniques [1.345251051985899]
This paper aims to perform sentiment analysis of real-time 2019 election twitter data using the feature selection model word2vec and the machine learning algorithm random forest for sentiment classification.
Word2vec improves the quality of features by considering contextual semantics of words in a text hence improving the accuracy of machine learning and sentiment analysis.
arXiv Detail & Related papers (2020-07-05T08:10:30Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.