Statistical Analysis of Perspective Scores on Hate Speech Detection

• URL: http://arxiv.org/abs/2107.02024v1
• Date: Tue, 22 Jun 2021 17:17:35 GMT
• Title: Statistical Analysis of Perspective Scores on Hate Speech Detection
• Authors: Hadi Mansourifar, Dana Alsagheer, Weidong Shi, Lan Ni, Yan Huang
• Abstract summary: State-of-the-art hate speech classifiers are efficient only when tested on the data with the same feature distribution as training data. In such a diverse data distribution relying on low level features is the main cause of deficiency due to natural bias in data. We show that, different hate speech datasets are very similar when it comes to extract their Perspective Scores.
• Score: 7.447951461558536
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Hate speech detection has become a hot topic in recent years due to the exponential growth of offensive language in social media. It has proven that, state-of-the-art hate speech classifiers are efficient only when tested on the data with the same feature distribution as training data. As a consequence, model architecture plays the second role to improve the current results. In such a diverse data distribution relying on low level features is the main cause of deficiency due to natural bias in data. That's why we need to use high level features to avoid a biased judgement. In this paper, we statistically analyze the Perspective Scores and their impact on hate speech detection. We show that, different hate speech datasets are very similar when it comes to extract their Perspective Scores. Eventually, we prove that, over-sampling the Perspective Scores of a hate speech dataset can significantly improve the generalization performance when it comes to be tested on other hate speech datasets.

Related papers

• A Target-Aware Analysis of Data Augmentation for Hate Speech Detection [3.858155067958448]
  Hate speech is one of the main threats posed by the widespread use of social networks. We investigate the possibility of augmenting existing data with generative language models, reducing target imbalance. For some hate categories such as origin, religion, and disability, hate speech classification using augmented data for training improves by more than 10% F1 over the no augmentation baseline.
  arXiv  Detail & Related papers  (2024-10-10T15:46:27Z)
• Hate Speech Detection Using Cross-Platform Social Media Data In English and German Language [6.200058263544999]
  This study focuses on detecting bilingual hate speech in YouTube comments. We include factors such as content similarity, definition similarity, and common hate words to measure the impact of datasets on performance. The best performance was obtained by combining datasets from YouTube comments, Twitter, and Gab with an F1-score of 0.74 and 0.68 for English and German YouTube comments.
  arXiv  Detail & Related papers  (2024-10-02T10:22:53Z)
• Into the LAIONs Den: Investigating Hate in Multimodal Datasets [67.21783778038645]
  This paper investigates the effect of scaling datasets on hateful content through a comparative audit of two datasets: LAION-400M and LAION-2B. We found that hate content increased by nearly 12% with dataset scale, measured both qualitatively and quantitatively. We also found that filtering dataset contents based on Not Safe For Work (NSFW) values calculated based on images alone does not exclude all the harmful content in alt-text.
  arXiv  Detail & Related papers  (2023-11-06T19:00:05Z)
• Robust Hate Speech Detection in Social Media: A Cross-Dataset Empirical Evaluation [5.16706940452805]
  We perform a large-scale cross-dataset comparison where we fine-tune language models on different hate speech detection datasets. This analysis shows how some datasets are more generalisable than others when used as training data. Experiments show how combining hate speech detection datasets can contribute to the development of robust hate speech detection models.
  arXiv  Detail & Related papers  (2023-07-04T12:22:40Z)
• ToKen: Task Decomposition and Knowledge Infusion for Few-Shot Hate Speech Detection [85.68684067031909]
  We frame this problem as a few-shot learning task, and show significant gains with decomposing the task into its "constituent" parts. In addition, we see that infusing knowledge from reasoning datasets (e.g. Atomic 2020) improves the performance even further.
  arXiv  Detail & Related papers  (2022-05-25T05:10:08Z)
• Deep Learning for Hate Speech Detection: A Comparative Study [54.42226495344908]
  We present here a large-scale empirical comparison of deep and shallow hate-speech detection methods. Our goal is to illuminate progress in the area, and identify strengths and weaknesses in the current state-of-the-art. In doing so we aim to provide guidance as to the use of hate-speech detection in practice, quantify the state-of-the-art, and identify future research directions.
  arXiv  Detail & Related papers  (2022-02-19T03:48:20Z)
• Addressing the Challenges of Cross-Lingual Hate Speech Detection [115.1352779982269]
  In this paper we focus on cross-lingual transfer learning to support hate speech detection in low-resource languages. We leverage cross-lingual word embeddings to train our neural network systems on the source language and apply it to the target language. We investigate the issue of label imbalance of hate speech datasets, since the high ratio of non-hate examples compared to hate examples often leads to low model performance.
  arXiv  Detail & Related papers  (2022-01-15T20:48:14Z)
• Reducing Target Group Bias in Hate Speech Detectors [56.94616390740415]
  We show that text classification models trained on large publicly available datasets, may significantly under-perform on several protected groups. We propose to perform token-level hate sense disambiguation, and utilize tokens' hate sense representations for detection.
  arXiv  Detail & Related papers  (2021-12-07T17:49:34Z)
• An Information Retrieval Approach to Building Datasets for Hate Speech Detection [3.587367153279349]
  A common practice is to only annotate tweets containing known hate words'' A second challenge is that definitions of hate speech tend to be highly variable and subjective. Our key insight is that the rarity and subjectivity of hate speech are akin to that of relevance in information retrieval (IR)
  arXiv  Detail & Related papers  (2021-06-17T19:25:39Z)
• Trawling for Trolling: A Dataset [56.1778095945542]
  We present a dataset that models trolling as a subcategory of offensive content. The dataset has 12,490 samples, split across 5 classes; Normal, Profanity, Trolling, Derogatory and Hate Speech.
  arXiv  Detail & Related papers  (2020-08-02T17:23:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.