Offensive Language Detection on Social Media Using XLNet

• URL: http://arxiv.org/abs/2506.21795v1
• Date: Thu, 26 Jun 2025 22:37:35 GMT
• Title: Offensive Language Detection on Social Media Using XLNet
• Authors: Reem Alothman, Hafida Benhidour, Said Kerrache,
• Abstract summary: We propose an automatic offensive language detection model based on XLNet, a generalized autoregressive pretraining method, and compare its performance with BERT (Bigressive Representations from Transformers)<n>Our experimental results show that XLNet outperforms BERT in detecting offensive content and in categorizing the types of offenses, while BERT performs slightly better in identifying the targets of the offenses.<n>These findings highlight the potential of transfer learning and XLNet-based architectures to create robust systems for detecting offensive language on social media platforms.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The widespread use of text-based communication on social media-through chats, comments, and microblogs-has improved user interaction but has also led to an increase in offensive content, including hate speech, racism, and other forms of abuse. Due to the enormous volume of user-generated content, manual moderation is impractical, which creates a need for automated systems that can detect offensive language. Deep learning models, particularly those using transfer learning, have demonstrated significant success in understanding natural language through large-scale pretraining. In this study, we propose an automatic offensive language detection model based on XLNet, a generalized autoregressive pretraining method, and compare its performance with BERT (Bidirectional Encoder Representations from Transformers), which is a widely used baseline in natural language processing (NLP). Both models are evaluated using the Offensive Language Identification Dataset (OLID), a benchmark Twitter dataset that includes hierarchical annotations. Our experimental results show that XLNet outperforms BERT in detecting offensive content and in categorizing the types of offenses, while BERT performs slightly better in identifying the targets of the offenses. Additionally, we find that oversampling and undersampling strategies are effective in addressing class imbalance and improving classification performance. These findings highlight the potential of transfer learning and XLNet-based architectures to create robust systems for detecting offensive language on social media platforms.

Related papers

• Developing Linguistic Patterns to Mitigate Inherent Human Bias in Offensive Language Detection [1.6574413179773761]
  We propose a linguistic data augmentation approach to reduce bias in labeling processes. This approach has the potential to improve offensive language classification tasks across multiple languages.
  arXiv  Detail & Related papers  (2023-12-04T10:20:36Z)
• Hate Speech and Offensive Language Detection using an Emotion-aware Shared Encoder [1.8734449181723825]
  Existing works on hate speech and offensive language detection produce promising results based on pre-trained transformer models. This paper addresses a multi-task joint learning approach which combines external emotional features extracted from another corpora. Our findings demonstrate that emotional knowledge helps to more reliably identify hate speech and offensive language across datasets.
  arXiv  Detail & Related papers  (2023-02-17T09:31:06Z)
• GanLM: Encoder-Decoder Pre-training with an Auxiliary Discriminator [114.8954615026781]
  We propose a GAN-style model for encoder-decoder pre-training by introducing an auxiliary discriminator. GanLM is trained with two pre-training objectives: replaced token detection and replaced token denoising. Experiments in language generation benchmarks show that GanLM with the powerful language understanding capability outperforms various strong pre-trained language models.
  arXiv  Detail & Related papers  (2022-12-20T12:51:11Z)
• Detecting Offensive Language on Social Networks: An End-to-end Detection Method based on Graph Attention Networks [7.723697303436006]
  We propose an end-to-end method based on community structure and text features for offensive language detection (CT-OLD) We add user opinion to the community structure for representing user features. The user opinion is represented by user historical behavior information, which outperforms that represented by text information.
  arXiv  Detail & Related papers  (2022-03-04T03:57:18Z)
• A New Generation of Perspective API: Efficient Multilingual Character-level Transformers [66.9176610388952]
  We present the fundamentals behind the next version of the Perspective API from Google Jigsaw. At the heart of the approach is a single multilingual token-free Charformer model. We demonstrate that by forgoing static vocabularies, we gain flexibility across a variety of settings.
  arXiv  Detail & Related papers  (2022-02-22T20:55:31Z)
• Improving Classifier Training Efficiency for Automatic Cyberbullying Detection with Feature Density [58.64907136562178]
  We study the effectiveness of Feature Density (FD) using different linguistically-backed feature preprocessing methods. We hypothesise that estimating dataset complexity allows for the reduction of the number of required experiments. The difference in linguistic complexity of datasets allows us to additionally discuss the efficacy of linguistically-backed word preprocessing.
  arXiv  Detail & Related papers  (2021-11-02T15:48:28Z)
• Offensive Language and Hate Speech Detection with Deep Learning and Transfer Learning [1.77356577919977]
  We propose an approach to automatically classify tweets into three classes: Hate, offensive and Neither. We create a class module which contains main functionality including text classification, sentiment checking and text data augmentation.
  arXiv  Detail & Related papers  (2021-08-06T20:59:47Z)
• Transfer Learning Approach for Arabic Offensive Language Detection System -- BERT-Based Model [0.0]
  Cyberhate, online harassment and other misuses of technology are on the rise. Applying advanced techniques from the Natural Language Processing (NLP) field to support the development of an online hate-free community is a critical task for social justice. This study aims at investigating the effects of fine-tuning and training Bidirectional Representations from Transformers (BERT) model on multiple Arabic offensive language datasets individually.
  arXiv  Detail & Related papers  (2021-02-09T04:58:18Z)
• InfoBERT: Improving Robustness of Language Models from An Information Theoretic Perspective [84.78604733927887]
  Large-scale language models such as BERT have achieved state-of-the-art performance across a wide range of NLP tasks. Recent studies show that such BERT-based models are vulnerable facing the threats of textual adversarial attacks. We propose InfoBERT, a novel learning framework for robust fine-tuning of pre-trained language models.
  arXiv  Detail & Related papers  (2020-10-05T20:49:26Z)
• How Context Affects Language Models' Factual Predictions [134.29166998377187]
  We integrate information from a retrieval system with a pre-trained language model in a purely unsupervised way. We report that augmenting pre-trained language models in this way dramatically improves performance and that the resulting system, despite being unsupervised, is competitive with a supervised machine reading baseline.
  arXiv  Detail & Related papers  (2020-05-10T09:28:12Z)
• Kungfupanda at SemEval-2020 Task 12: BERT-Based Multi-Task Learning for Offensive Language Detection [55.445023584632175]
  We build an offensive language detection system, which combines multi-task learning with BERT-based models. Our model achieves 91.51% F1 score in English Sub-task A, which is comparable to the first place.
  arXiv  Detail & Related papers  (2020-04-28T11:27:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.