A hybrid transformer and attention based recurrent neural network for robust and interpretable sentiment analysis of tweets

• URL: http://arxiv.org/abs/2404.00297v5
• Date: Sat, 02 Nov 2024 18:03:03 GMT
• Title: A hybrid transformer and attention based recurrent neural network for robust and interpretable sentiment analysis of tweets
• Authors: Md Abrar Jahin, Md Sakib Hossain Shovon, M. F. Mridha, Md Rashedul Islam, Yutaka Watanobe,
• Abstract summary: Existing models face challenges with linguistic diversity, generalizability, and explainability. We propose TRABSA, a hybrid framework integrating transformer-based architectures, attention mechanisms, and BiLSTM networks. We bridge gaps in sentiment analysis benchmarks, ensuring state-of-the-art accuracy.
• Score: 0.3495246564946556
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Sentiment analysis is crucial for understanding public opinion and consumer behavior. Existing models face challenges with linguistic diversity, generalizability, and explainability. We propose TRABSA, a hybrid framework integrating transformer-based architectures, attention mechanisms, and BiLSTM networks to address this. Leveraging RoBERTa-trained on 124M tweets, we bridge gaps in sentiment analysis benchmarks, ensuring state-of-the-art accuracy. Augmenting datasets with tweets from 32 countries and US states, we compare six word-embedding techniques and three lexicon-based labeling techniques, selecting the best for optimal sentiment analysis. TRABSA outperforms traditional ML and deep learning models with 94% accuracy and significant precision, recall, and F1-score gains. Evaluation across diverse datasets demonstrates consistent superiority and generalizability. SHAP and LIME analyses enhance interpretability, improving confidence in predictions. Our study facilitates pandemic resource management, aiding resource planning, policy formation, and vaccination tactics.

Related papers

• TWSSenti: A Novel Hybrid Framework for Topic-Wise Sentiment Analysis on Social Media Using Transformer Models [0.0]
  This study explores a hybrid framework combining transformer-based models to improve sentiment classification accuracy and robustness. The framework addresses challenges such as noisy data, contextual ambiguity, and generalization across diverse datasets. This research highlights its applicability to real-world tasks such as social media monitoring, customer sentiment analysis, and public opinion tracking.
  arXiv  Detail & Related papers  (2025-04-14T05:44:11Z)
• Sentiment analysis of texts from social networks based on machine learning methods for monitoring public sentiment [0.0]
  A sentiment analysis system powered by machine learning was created in this study to improve real-time social network public opinion monitoring. The system achieved an accuracy of up to 80-85% using transformer models in real-world scenarios. Despite the system's impressive performance, issues with computing overhead, data quality, and domain-specific terminology still exist.
  arXiv  Detail & Related papers  (2025-02-24T13:34:35Z)
• Three-Class Text Sentiment Analysis Based on LSTM [0.0]
  This paper introduces a three-class sentiment classification method for Weibo comments using Long Short-Term Memory (LSTM) networks. Experimental results demonstrate superior performance, achieving an accuracy of 98.31% and an F1 score of 98.28%.
  arXiv  Detail & Related papers  (2024-12-23T07:21:07Z)
• Bias-Free Sentiment Analysis through Semantic Blinding and Graph Neural Networks [0.0]
  The SProp GNN relies exclusively on syntactic structures and word-level emotional cues to predict emotions in text. By semantically blinding the model to information about specific words, it is robust to biases such as political or gender bias. The SProp GNN shows performance superior to lexicon-based alternatives on two different prediction tasks, and across two languages.
  arXiv  Detail & Related papers  (2024-11-19T13:23:53Z)
• What Do Learning Dynamics Reveal About Generalization in LLM Reasoning? [83.83230167222852]
  We find that a model's generalization behavior can be effectively characterized by a training metric we call pre-memorization train accuracy. By connecting a model's learning behavior to its generalization, pre-memorization train accuracy can guide targeted improvements to training strategies.
  arXiv  Detail & Related papers  (2024-11-12T09:52:40Z)
• Improving Network Interpretability via Explanation Consistency Evaluation [56.14036428778861]
  We propose a framework that acquires more explainable activation heatmaps and simultaneously increase the model performance. Specifically, our framework introduces a new metric, i.e., explanation consistency, to reweight the training samples adaptively in model learning. Our framework then promotes the model learning by paying closer attention to those training samples with a high difference in explanations.
  arXiv  Detail & Related papers  (2024-08-08T17:20:08Z)
• RoBERTa-BiLSTM: A Context-Aware Hybrid Model for Sentiment Analysis [0.0]
  We introduce a novel hybrid deep learning model, RoBERTa-BiLSTM, which combines the Robustly Optimized BERT Pretraining Approach (RoBERTa) with Bi Long Short-Term Memory (BiLSTM) networks. RoBERTa is utilized to generate meaningful word embedding vectors, while BiLSTM effectively captures the contextual semantics of long-dependent texts. We conducted experiments using datasets from IMDb, Twitter US Airline, and Sentiment140 to evaluate the proposed model against existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-06-01T08:59:46Z)
• Adversarial Capsule Networks for Romanian Satire Detection and Sentiment Analysis [0.13048920509133807]
  Satire detection and sentiment analysis are intensively explored natural language processing tasks. In languages with fewer research resources, an alternative is to produce artificial examples based on character-level adversarial processes. In this work, we improve the well-known NLP models with adversarial training and capsule networks. The proposed framework outperforms the existing methods for the two tasks, achieving up to 99.08% accuracy.
  arXiv  Detail & Related papers  (2023-06-13T15:23:44Z)
• On the Robustness of Aspect-based Sentiment Analysis: Rethinking Model, Data, and Training [109.9218185711916]
  Aspect-based sentiment analysis (ABSA) aims at automatically inferring the specific sentiment polarities toward certain aspects of products or services behind social media texts or reviews. We propose to enhance the ABSA robustness by systematically rethinking the bottlenecks from all possible angles, including model, data, and training.
  arXiv  Detail & Related papers  (2023-04-19T11:07:43Z)
• Semantic Image Attack for Visual Model Diagnosis [80.36063332820568]
  In practice, metric analysis on a specific train and test dataset does not guarantee reliable or fair ML models. This paper proposes Semantic Image Attack (SIA), a method based on the adversarial attack that provides semantic adversarial images.
  arXiv  Detail & Related papers  (2023-03-23T03:13:04Z)
• Aspect-Based Sentiment Analysis using Local Context Focus Mechanism with DeBERTa [23.00810941211685]
  Aspect-Based Sentiment Analysis (ABSA) is a fine-grained task in the field of sentiment analysis. Recent DeBERTa model (Decoding-enhanced BERT with disentangled attention) to solve Aspect-Based Sentiment Analysis problem.
  arXiv  Detail & Related papers  (2022-07-06T03:50:31Z)
• Incorporating Dynamic Semantics into Pre-Trained Language Model for Aspect-based Sentiment Analysis [67.41078214475341]
  We propose Dynamic Re-weighting BERT (DR-BERT) to learn dynamic aspect-oriented semantics for ABSA. Specifically, we first take the Stack-BERT layers as a primary encoder to grasp the overall semantic of the sentence. We then fine-tune it by incorporating a lightweight Dynamic Re-weighting Adapter (DRA)
  arXiv  Detail & Related papers  (2022-03-30T14:48:46Z)
• Vision Transformers are Robust Learners [65.91359312429147]
  We study the robustness of the Vision Transformer (ViT) against common corruptions and perturbations, distribution shifts, and natural adversarial examples. We present analyses that provide both quantitative and qualitative indications to explain why ViTs are indeed more robust learners.
  arXiv  Detail & Related papers  (2021-05-17T02:39:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.