Performance evaluation of Reddit Comments using Machine Learning and Natural Language Processing methods in Sentiment Analysis

• URL: http://arxiv.org/abs/2405.16810v2
• Date: Tue, 28 May 2024 14:28:49 GMT
• Title: Performance evaluation of Reddit Comments using Machine Learning and Natural Language Processing methods in Sentiment Analysis
• Authors: Xiaoxia Zhang, Xiuyuan Qi, Zixin Teng,
• Abstract summary: We evaluate sentiment analysis methods across a corpus of 58,000 comments on Reddit. Our research expands the scope by evaluating a diverse array of models. Our findings reveal that the RoBERTa model consistently outperforms the baseline models.
• Score: 0.764671395172401
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Sentiment analysis, an increasingly vital field in both academia and industry, plays a pivotal role in machine learning applications, particularly on social media platforms like Reddit. However, the efficacy of sentiment analysis models is hindered by the lack of expansive and fine-grained emotion datasets. To address this gap, our study leverages the GoEmotions dataset, comprising a diverse range of emotions, to evaluate sentiment analysis methods across a substantial corpus of 58,000 comments. Distinguished from prior studies by the Google team, which limited their analysis to only two models, our research expands the scope by evaluating a diverse array of models. We investigate the performance of traditional classifiers such as Naive Bayes and Support Vector Machines (SVM), as well as state-of-the-art transformer-based models including BERT, RoBERTa, and GPT. Furthermore, our evaluation criteria extend beyond accuracy to encompass nuanced assessments, including hierarchical classification based on varying levels of granularity in emotion categorization. Additionally, considerations such as computational efficiency are incorporated to provide a comprehensive evaluation framework. Our findings reveal that the RoBERTa model consistently outperforms the baseline models, demonstrating superior accuracy in fine-grained sentiment classification tasks. This underscores the substantial potential and significance of the RoBERTa model in advancing sentiment analysis capabilities.

Related papers

• GenBench: A Benchmarking Suite for Systematic Evaluation of Genomic Foundation Models [56.63218531256961]
  We introduce GenBench, a benchmarking suite specifically tailored for evaluating the efficacy of Genomic Foundation Models. GenBench offers a modular and expandable framework that encapsulates a variety of state-of-the-art methodologies. We provide a nuanced analysis of the interplay between model architecture and dataset characteristics on task-specific performance.
  arXiv  Detail & Related papers  (2024-06-01T08:01:05Z)
• Finding fake reviews in e-commerce platforms by using hybrid algorithms [0.0]
  We propose an innovative ensemble approach for sentiment analysis for finding fake reviews. Our ensemble architecture strategically combines diverse models to capitalize on their strengths while mitigating inherent weaknesses. Our findings underscore the potential of ensemble techniques in advancing the state-of-the-art in finding fake reviews.
  arXiv  Detail & Related papers  (2024-04-09T14:25:27Z)
• Predictive Analytics of Varieties of Potatoes [2.336821989135698]
  We explore the application of machine learning algorithms to predict the suitability of Russet potato clones for advancement in breeding trials. We leverage data from manually collected trials in the state of Oregon.
  arXiv  Detail & Related papers  (2024-04-04T00:49:05Z)
• F-Eval: Asssessing Fundamental Abilities with Refined Evaluation Methods [111.46455901113976]
  We propose F-Eval, a bilingual evaluation benchmark to evaluate the fundamental abilities, including expression, commonsense and logic. For reference-free subjective tasks, we devise new evaluation methods, serving as alternatives to scoring by API models.
  arXiv  Detail & Related papers  (2024-01-26T13:55:32Z)
• Exploring the Power of Topic Modeling Techniques in Analyzing Customer Reviews: A Comparative Analysis [0.0]
  Machine learning and natural language processing algorithms have been deployed to analyze the vast amount of textual data available online. In this study, we examine and compare five frequently used topic modeling methods specifically applied to customer reviews. Our findings reveal that BERTopic consistently yield more meaningful extracted topics and achieve favorable results.
  arXiv  Detail & Related papers  (2023-08-19T08:18:04Z)
• Evaluating the Generation Capabilities of Large Chinese Language Models [27.598864484231477]
  This paper unveils CG-Eval, the first-ever comprehensive and automated evaluation framework. It assesses the generative capabilities of large Chinese language models across a spectrum of academic disciplines. Gscore automates the quality measurement of a model's text generation against reference standards.
  arXiv  Detail & Related papers  (2023-08-09T09:22:56Z)
• Aspect-Based Sentiment Analysis using Local Context Focus Mechanism with DeBERTa [23.00810941211685]
  Aspect-Based Sentiment Analysis (ABSA) is a fine-grained task in the field of sentiment analysis. Recent DeBERTa model (Decoding-enhanced BERT with disentangled attention) to solve Aspect-Based Sentiment Analysis problem.
  arXiv  Detail & Related papers  (2022-07-06T03:50:31Z)
• A comprehensive comparative evaluation and analysis of Distributional Semantic Models [61.41800660636555]
  We perform a comprehensive evaluation of type distributional vectors, either produced by static DSMs or obtained by averaging the contextualized vectors generated by BERT. The results show that the alleged superiority of predict based models is more apparent than real, and surely not ubiquitous. We borrow from cognitive neuroscience the methodology of Representational Similarity Analysis (RSA) to inspect the semantic spaces generated by distributional models.
  arXiv  Detail & Related papers  (2021-05-20T15:18:06Z)
• RADDLE: An Evaluation Benchmark and Analysis Platform for Robust Task-oriented Dialog Systems [75.87418236410296]
  We introduce the RADDLE benchmark, a collection of corpora and tools for evaluating the performance of models across a diverse set of domains. RADDLE is designed to favor and encourage models with a strong generalization ability. We evaluate recent state-of-the-art systems based on pre-training and fine-tuning, and find that grounded pre-training on heterogeneous dialog corpora performs better than training a separate model per domain.
  arXiv  Detail & Related papers  (2020-12-29T08:58:49Z)
• Interpretable Multi-dataset Evaluation for Named Entity Recognition [110.64368106131062]
  We present a general methodology for interpretable evaluation for the named entity recognition (NER) task. The proposed evaluation method enables us to interpret the differences in models and datasets, as well as the interplay between them. By making our analysis tool available, we make it easy for future researchers to run similar analyses and drive progress in this area.
  arXiv  Detail & Related papers  (2020-11-13T10:53:27Z)
• Rethinking Generalization of Neural Models: A Named Entity Recognition Case Study [81.11161697133095]
  We take the NER task as a testbed to analyze the generalization behavior of existing models from different perspectives. Experiments with in-depth analyses diagnose the bottleneck of existing neural NER models. As a by-product of this paper, we have open-sourced a project that involves a comprehensive summary of recent NER papers.
  arXiv  Detail & Related papers  (2020-01-12T04:33:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.