Semantic and Contextual Modeling for Malicious Comment Detection with BERT-BiLSTM

• URL: http://arxiv.org/abs/2503.11084v1
• Date: Fri, 14 Mar 2025 04:51:36 GMT
• Title: Semantic and Contextual Modeling for Malicious Comment Detection with BERT-BiLSTM
• Authors: Zhou Fang, Hanlu Zhang, Jacky He, Zhen Qi, Hongye Zheng,
• Abstract summary: We propose a deep learning model that combines BERT and BiLSTM.<n>The BERT model, through pre-training, captures deep semantic features of text, while the BiLSTM network excels at processing sequential data.<n> Experimental results on the Jigsaw Unintended Bias in Toxicity Classification dataset demonstrate that the BERT+BiLSTM model achieves superior performance in malicious comment detection tasks.
• Score: 1.5845117761091052
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This study aims to develop an efficient and accurate model for detecting malicious comments, addressing the increasingly severe issue of false and harmful content on social media platforms. We propose a deep learning model that combines BERT and BiLSTM. The BERT model, through pre-training, captures deep semantic features of text, while the BiLSTM network excels at processing sequential data and can further model the contextual dependencies of text. Experimental results on the Jigsaw Unintended Bias in Toxicity Classification dataset demonstrate that the BERT+BiLSTM model achieves superior performance in malicious comment detection tasks, with a precision of 0.94, recall of 0.93, and accuracy of 0.94. This surpasses other models, including standalone BERT, TextCNN, TextRNN, and traditional machine learning algorithms using TF-IDF features. These results confirm the superiority of the BERT+BiLSTM model in handling imbalanced data and capturing deep semantic features of malicious comments, providing an effective technical means for social media content moderation and online environment purification.

Related papers

• TWSSenti: A Novel Hybrid Framework for Topic-Wise Sentiment Analysis on Social Media Using Transformer Models [0.0]
  This study explores a hybrid framework combining transformer-based models to improve sentiment classification accuracy and robustness. The framework addresses challenges such as noisy data, contextual ambiguity, and generalization across diverse datasets. This research highlights its applicability to real-world tasks such as social media monitoring, customer sentiment analysis, and public opinion tracking.
  arXiv  Detail & Related papers  (2025-04-14T05:44:11Z)
• A Trustable LSTM-Autoencoder Network for Cyberbullying Detection on Social Media Using Synthetic Data [2.378735224874938]
  This paper proposes a trustable LSTM-Autoencoder Network for cyberbullying detection on social media. We have demonstrated a cutting-edge method to address data availability difficulties by producing machine-translated data. We carried out experimental identification of aggressive comments on Hindi, Bangla, and English datasets.
  arXiv  Detail & Related papers  (2023-08-15T17:20:05Z)
• Adapted Multimodal BERT with Layer-wise Fusion for Sentiment Analysis [84.12658971655253]
  We propose Adapted Multimodal BERT, a BERT-based architecture for multimodal tasks. adapter adjusts the pretrained language model for the task at hand, while the fusion layers perform task-specific, layer-wise fusion of audio-visual information with textual BERT representations. In our ablations we see that this approach leads to efficient models, that can outperform their fine-tuned counterparts and are robust to input noise.
  arXiv  Detail & Related papers  (2022-12-01T17:31:42Z)
• BiBERT: Accurate Fully Binarized BERT [69.35727280997617]
  BiBERT is an accurate fully binarized BERT to eliminate the performance bottlenecks. Our method yields impressive 56.3 times and 31.2 times saving on FLOPs and model size.
  arXiv  Detail & Related papers  (2022-03-12T09:46:13Z)
• On Sensitivity of Deep Learning Based Text Classification Algorithms to Practical Input Perturbations [0.0]
  We evaluate the impact of systematic practical perturbations on the performance of deep learning based text classification models. The perturbations are induced by the addition and removal of unwanted tokens like punctuation and stop-words. We show that these deep learning approaches including BERT are sensitive to such legitimate input perturbations on four standard benchmark datasets.
  arXiv  Detail & Related papers  (2022-01-02T08:33:49Z)
• Deploying a BERT-based Query-Title Relevance Classifier in a Production System: a View from the Trenches [3.1219977244201056]
  Bidirectional Representations from Transformers (BERT) model has been radically improving the performance of many Natural Language Processing (NLP) tasks. It is challenging to scale BERT for low-latency and high- throughput industrial use cases due to its enormous size. We successfully optimize a Query-Title Relevance (QTR) classifier for deployment via a compact model, which we name BERT Bidirectional Long Short-Term Memory (BertBiLSTM) BertBiLSTM exceeds the off-the-shelf BERT model's performance in terms of accuracy and efficiency for the aforementioned real-world production task
  arXiv  Detail & Related papers  (2021-08-23T14:28:23Z)
• MATE-KD: Masked Adversarial TExt, a Companion to Knowledge Distillation [9.91548921801095]
  We present, MATE-KD, a novel text-based adversarial training algorithm which improves the performance of knowledge distillation. We evaluate our algorithm, using BERT-based models, on the GLUE benchmark and demonstrate that MATE-KD outperforms competitive adversarial learning and data augmentation baselines.
  arXiv  Detail & Related papers  (2021-05-12T19:11:34Z)
• Evaluation of BERT and ALBERT Sentence Embedding Performance on Downstream NLP Tasks [4.955649816620742]
  This paper explores on sentence embedding models for BERT and ALBERT. We take a modified BERT network with siamese and triplet network structures called Sentence-BERT (SBERT) and replace BERT with ALBERT to create Sentence-ALBERT (SALBERT)
  arXiv  Detail & Related papers  (2021-01-26T09:14:06Z)
• Incorporating BERT into Parallel Sequence Decoding with Adapters [82.65608966202396]
  We propose to take two different BERT models as the encoder and decoder respectively, and fine-tune them by introducing simple and lightweight adapter modules. We obtain a flexible and efficient model which is able to jointly leverage the information contained in the source-side and target-side BERT models. Our framework is based on a parallel sequence decoding algorithm named Mask-Predict considering the bi-directional and conditional independent nature of BERT.
  arXiv  Detail & Related papers  (2020-10-13T03:25:15Z)
• InfoBERT: Improving Robustness of Language Models from An Information Theoretic Perspective [84.78604733927887]
  Large-scale language models such as BERT have achieved state-of-the-art performance across a wide range of NLP tasks. Recent studies show that such BERT-based models are vulnerable facing the threats of textual adversarial attacks. We propose InfoBERT, a novel learning framework for robust fine-tuning of pre-trained language models.
  arXiv  Detail & Related papers  (2020-10-05T20:49:26Z)
• Revisiting LSTM Networks for Semi-Supervised Text Classification via Mixed Objective Function [106.69643619725652]
  We develop a training strategy that allows even a simple BiLSTM model, when trained with cross-entropy loss, to achieve competitive results. We report state-of-the-art results for text classification task on several benchmark datasets.
  arXiv  Detail & Related papers  (2020-09-08T21:55:22Z)
• BERT-ATTACK: Adversarial Attack Against BERT Using BERT [77.82947768158132]
  Adrial attacks for discrete data (such as texts) are more challenging than continuous data (such as images) We propose textbfBERT-Attack, a high-quality and effective method to generate adversarial samples. Our method outperforms state-of-the-art attack strategies in both success rate and perturb percentage.
  arXiv  Detail & Related papers  (2020-04-21T13:30:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.