Fine-tuning BERT with Bidirectional LSTM for Fine-grained Movie Reviews Sentiment Analysis

• URL: http://arxiv.org/abs/2502.20682v1
• Date: Fri, 28 Feb 2025 03:30:48 GMT
• Title: Fine-tuning BERT with Bidirectional LSTM for Fine-grained Movie Reviews Sentiment Analysis
• Authors: Gibson Nkhata, Susan Gauch, Usman Anjum, Justin Zhan,
• Abstract summary: We fine-tune the pre-trained BERT model with Bidirectional LSTM (BiLSTM) to enhance both binary and fine-grained SA for movie reviews.<n>We present our findings on binary classification as well as fine-grained classification utilizing benchmark datasets.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Sentiment Analysis (SA) is instrumental in understanding peoples viewpoints facilitating social media monitoring recognizing products and brands and gauging customer satisfaction. Consequently SA has evolved into an active research domain within Natural Language Processing (NLP). Many approaches outlined in the literature devise intricate frameworks aimed at achieving high accuracy, focusing exclusively on either binary sentiment classification or fine-grained sentiment classification. In this paper our objective is to fine-tune the pre-trained BERT model with Bidirectional LSTM (BiLSTM) to enhance both binary and fine-grained SA specifically for movie reviews. Our approach involves conducting sentiment classification for each review followed by computing the overall sentiment polarity across all reviews. We present our findings on binary classification as well as fine-grained classification utilizing benchmark datasets. Additionally we implement and assess two accuracy improvement techniques Synthetic Minority Oversampling Technique (SMOTE) and NLP Augmenter (NLPAUG) to bolster the models generalization in fine-grained sentiment classification. Finally a heuristic algorithm is employed to calculate the overall polarity of predicted reviews from the BERT+BiLSTM output vector. Our approach performs comparably with state-of-the-art (SOTA) techniques in both classifications. For instance in binary classification we achieve 97.67% accuracy surpassing the leading SOTA model NB-weighted-BON+dv-cosine by 0.27% on the renowned IMDb dataset. Conversely for five-class classification on SST-5 while the top SOTA model RoBERTa+large+Self-explaining attains 55.5% accuracy our model achieves 59.48% accuracy surpassing the BERT-large baseline by 3.6%.

Related papers

• Adaptive Cyber-Attack Detection in IIoT Using Attention-Based LSTM-CNN Models [0.23408308015481666]
  This study presents the development and evaluation of an advanced Intrusion detection (IDS) based on a hybrid LSTM-convolution neural network (CNN)-Attention architecture.<n>The research focuses on two key classification tasks: binary and multi-class classification.<n>In binary classification, the model achieved near-perfect accuracy, while in multi-class classification, it maintained a high accuracy level (99.04%), effectively categorizing different attack types with a loss value of 0.0220%.
  arXiv  Detail & Related papers  (2025-01-21T20:52:23Z)
• Multitask Fine-Tuning and Generative Adversarial Learning for Improved Auxiliary Classification [0.0]
  We implement a novel BERT architecture for multitask fine-tuning on three downstream tasks. Our model, Multitask BERT, incorporates layer sharing and a triplet architecture, custom sentence pair tokenization, loss pairing, and gradient surgery. We also apply generative adversarial learning to BERT, constructing a conditional generator model that maps from latent space to create fake embeddings.
  arXiv  Detail & Related papers  (2024-08-11T20:05:54Z)
• SAM as the Guide: Mastering Pseudo-Label Refinement in Semi-Supervised Referring Expression Segmentation [66.92696817276288]
  SemiRES is a semi-supervised framework that effectively leverages a combination of labeled and unlabeled data to perform RES. SemiRES incorporates the Segment Anything Model (SAM), renowned for its precise boundary demarcation. In instances where a precise mask cannot be matched from the available candidates, we develop the Pixel-Wise Adjustment (PWA) strategy.
  arXiv  Detail & Related papers  (2024-06-03T15:42:30Z)
• An Evaluation of Machine Learning Approaches for Early Diagnosis of Autism Spectrum Disorder [0.0]
  Autistic Spectrum Disorder (ASD) is a neurological disease characterized by difficulties with social interaction, communication, and repetitive activities. This study employs diverse machine learning methods to identify crucial ASD traits, aiming to enhance and automate the diagnostic process.
  arXiv  Detail & Related papers  (2023-09-20T21:23:37Z)
• Pretraining Without Attention [114.99187017618408]
  This work explores pretraining without attention by using recent advances in sequence routing based on state-space models (SSMs) BiGS is able to match BERT pretraining accuracy on GLUE and can be extended to long-form pretraining of 4096 tokens without approximation.
  arXiv  Detail & Related papers  (2022-12-20T18:50:08Z)
• Towards Semi-Supervised Deep Facial Expression Recognition with An Adaptive Confidence Margin [92.76372026435858]
  We learn an Adaptive Confidence Margin (Ada-CM) to fully leverage all unlabeled data for semi-supervised deep facial expression recognition. All unlabeled samples are partitioned into two subsets by comparing their confidence scores with the adaptively learned confidence margin. Our method achieves state-of-the-art performance, especially surpassing fully-supervised baselines in a semi-supervised manner.
  arXiv  Detail & Related papers  (2022-03-23T11:43:29Z)
• Delving into Probabilistic Uncertainty for Unsupervised Domain Adaptive Person Re-Identification [54.174146346387204]
  We propose an approach named probabilistic uncertainty guided progressive label refinery (P$2$LR) for domain adaptive person re-identification. A quantitative criterion is established to measure the uncertainty of pseudo labels and facilitate the network training. Our method outperforms the baseline by 6.5% mAP on the Duke2Market task, while surpassing the state-of-the-art method by 2.5% mAP on the Market2MSMT task.
  arXiv  Detail & Related papers  (2021-12-28T07:40:12Z)
• SCARF: Self-Supervised Contrastive Learning using Random Feature Corruption [72.35532598131176]
  We propose SCARF, a technique for contrastive learning, where views are formed by corrupting a random subset of features. We show that SCARF complements existing strategies and outperforms alternatives like autoencoders.
  arXiv  Detail & Related papers  (2021-06-29T08:08:33Z)
• NLRG at SemEval-2021 Task 5: Toxic Spans Detection Leveraging BERT-based Token Classification and Span Prediction Techniques [0.6850683267295249]
  In this paper, we explore simple versions of Token Classification or Span Prediction approaches. We use BERT-based models -- BERT, RoBERTa, and SpanBERT for both approaches. To this end, we investigate results on four hybrid approaches -- Multi-Span, Span+Token, LSTM-CRF, and a combination of predicted offsets using union/intersection.
  arXiv  Detail & Related papers  (2021-02-24T12:30:09Z)
• Revisiting LSTM Networks for Semi-Supervised Text Classification via Mixed Objective Function [106.69643619725652]
  We develop a training strategy that allows even a simple BiLSTM model, when trained with cross-entropy loss, to achieve competitive results. We report state-of-the-art results for text classification task on several benchmark datasets.
  arXiv  Detail & Related papers  (2020-09-08T21:55:22Z)
• Language Representation Models for Fine-Grained Sentiment Classification [2.1664197735413824]
  We show that AlBERT suffers significantly more accuracy loss than reported on other tasks, while DistilBERT has accuracy loss similar to their reported loss on other tasks. We conclude that RoBERTa reaches anew state-of-the-art accuracy for prediction on the SST-5 root level (60.2%).
  arXiv  Detail & Related papers  (2020-05-27T20:01:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.