HARE: Explainable Hate Speech Detection with Step-by-Step Reasoning

• URL: http://arxiv.org/abs/2311.00321v2
• Date: Wed, 22 Nov 2023 09:08:03 GMT
• Title: HARE: Explainable Hate Speech Detection with Step-by-Step Reasoning
• Authors: Yongjin Yang, Joonkee Kim, Yujin Kim, Namgyu Ho, James Thorne, Se-young Yun
• Abstract summary: We introduce a hate speech detection framework, HARE, which harnesses the reasoning capabilities of large language models (LLMs) to fill gaps in explanations of hate speech. Experiments on SBIC and Implicit Hate benchmarks show that our method, using model-generated data, consistently outperforms baselines. Our method enhances the explanation quality of trained models and improves generalization to unseen datasets.
• Score: 29.519687405350304
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: With the proliferation of social media, accurate detection of hate speech has become critical to ensure safety online. To combat nuanced forms of hate speech, it is important to identify and thoroughly explain hate speech to help users understand its harmful effects. Recent benchmarks have attempted to tackle this issue by training generative models on free-text annotations of implications in hateful text. However, we find significant reasoning gaps in the existing annotations schemes, which may hinder the supervision of detection models. In this paper, we introduce a hate speech detection framework, HARE, which harnesses the reasoning capabilities of large language models (LLMs) to fill these gaps in explanations of hate speech, thus enabling effective supervision of detection models. Experiments on SBIC and Implicit Hate benchmarks show that our method, using model-generated data, consistently outperforms baselines, using existing free-text human annotations. Analysis demonstrates that our method enhances the explanation quality of trained models and improves generalization to unseen datasets. Our code is available at https://github.com/joonkeekim/hare-hate-speech.git.

Related papers

• Towards Interpretable Hate Speech Detection using Large Language Model-extracted Rationales [15.458557611029518]
  Social media platforms are a prominent arena for users to engage in interpersonal discussions and express opinions. There arises a need to automatically identify and flag instances of hate speech. We propose to use state-of-the-art Large Language Models (LLMs) to extract features in the form of rationales from the input text.
  arXiv  Detail & Related papers  (2024-03-19T03:22:35Z)
• GPT-HateCheck: Can LLMs Write Better Functional Tests for Hate Speech Detection? [50.53312866647302]
  HateCheck is a suite for testing fine-grained model functionalities on synthesized data. We propose GPT-HateCheck, a framework to generate more diverse and realistic functional tests from scratch. Crowd-sourced annotation demonstrates that the generated test cases are of high quality.
  arXiv  Detail & Related papers  (2024-02-23T10:02:01Z)
• An Investigation of Large Language Models for Real-World Hate Speech Detection [46.15140831710683]
  A major limitation of existing methods is that hate speech detection is a highly contextual problem. Recently, large language models (LLMs) have demonstrated state-of-the-art performance in several natural language tasks. Our study reveals that a meticulously crafted reasoning prompt can effectively capture the context of hate speech.
  arXiv  Detail & Related papers  (2024-01-07T00:39:33Z)
• Hate Speech Detection in Limited Data Contexts using Synthetic Data Generation [1.9506923346234724]
  We propose a data augmentation approach that addresses the problem of lack of data for online hate speech detection in limited data contexts. We present three methods to synthesize new examples of hate speech data in a target language that retains the hate sentiment in the original examples but transfers the hate targets. Our findings show that a model trained on synthetic data performs comparably to, and in some cases outperforms, a model trained only on the samples available in the target domain.
  arXiv  Detail & Related papers  (2023-10-04T15:10:06Z)
• Verifying the Robustness of Automatic Credibility Assessment [79.08422736721764]
  Text classification methods have been widely investigated as a way to detect content of low credibility. In some cases insignificant changes in input text can mislead the models. We introduce BODEGA: a benchmark for testing both victim models and attack methods on misinformation detection tasks.
  arXiv  Detail & Related papers  (2023-03-14T16:11:47Z)
• CoSyn: Detecting Implicit Hate Speech in Online Conversations Using a Context Synergized Hyperbolic Network [52.85130555886915]
  CoSyn is a context-synergized neural network that explicitly incorporates user- and conversational context for detecting implicit hate speech in online conversations. We show that CoSyn outperforms all our baselines in detecting implicit hate speech with absolute improvements in the range of 1.24% - 57.8%.
  arXiv  Detail & Related papers  (2023-03-02T17:30:43Z)
• Leveraging World Knowledge in Implicit Hate Speech Detection [5.5536024561229205]
  We show that real world knowledge about entity mentions in a text does help models better detect hate speech. We also discuss cases where real world knowledge does not add value to hate speech detection.
  arXiv  Detail & Related papers  (2022-12-28T21:23:55Z)
• Deep Learning for Hate Speech Detection: A Comparative Study [54.42226495344908]
  We present here a large-scale empirical comparison of deep and shallow hate-speech detection methods. Our goal is to illuminate progress in the area, and identify strengths and weaknesses in the current state-of-the-art. In doing so we aim to provide guidance as to the use of hate-speech detection in practice, quantify the state-of-the-art, and identify future research directions.
  arXiv  Detail & Related papers  (2022-02-19T03:48:20Z)
• Addressing the Challenges of Cross-Lingual Hate Speech Detection [115.1352779982269]
  In this paper we focus on cross-lingual transfer learning to support hate speech detection in low-resource languages. We leverage cross-lingual word embeddings to train our neural network systems on the source language and apply it to the target language. We investigate the issue of label imbalance of hate speech datasets, since the high ratio of non-hate examples compared to hate examples often leads to low model performance.
  arXiv  Detail & Related papers  (2022-01-15T20:48:14Z)
• Latent Hatred: A Benchmark for Understanding Implicit Hate Speech [22.420275418616242]
  This work introduces a theoretically-justified taxonomy of implicit hate speech and a benchmark corpus with fine-grained labels for each message. We present systematic analyses of our dataset using contemporary baselines to detect and explain implicit hate speech.
  arXiv  Detail & Related papers  (2021-09-11T16:52:56Z)
• Towards Hate Speech Detection at Large via Deep Generative Modeling [4.080068044420974]
  Hate speech detection is a critical problem in social media platforms. We present a dataset of 1 million realistic hate and non-hate sequences, produced by a deep generative language model. We demonstrate consistent and significant performance improvements across five public hate speech datasets.
  arXiv  Detail & Related papers  (2020-05-13T15:25:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.