When Hate Meets Facts: LLMs-in-the-Loop for Check-worthiness Detection in Hate Speech

• URL: http://arxiv.org/abs/2603.25269v1
• Date: Thu, 26 Mar 2026 10:07:00 GMT
• Title: When Hate Meets Facts: LLMs-in-the-Loop for Check-worthiness Detection in Hate Speech
• Abstract summary: We release WSF-ARG+, the first dataset which combines hate speech with check-worthiness information.<n>We introduce a novel LLM-in-the-loop framework to facilitate the annotation of check-worthy claims.<n>We show that HS messages with check-worthy claims show significantly higher harassment and hate.
• Score: 4.736364625993341
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Hateful content online is often expressed using fact-like, not necessarily correct information, especially in coordinated online harassment campaigns and extremist propaganda. Failing to jointly address hate speech (HS) and misinformation can deepen prejudice, reinforce harmful stereotypes, and expose bystanders to psychological distress, while polluting public debate. Moreover, these messages require more effort from content moderators because they must assess both harmfulness and veracity, i.e., fact-check them. To address this challenge, we release WSF-ARG+, the first dataset which combines hate speech with check-worthiness information. We also introduce a novel LLM-in-the-loop framework to facilitate the annotation of check-worthy claims. We run our framework, testing it with 12 open-weight LLMs of different sizes and architectures. We validate it through extensive human evaluation, and show that our LLM-in-the-loop framework reduces human effort without compromising the annotation quality of the data. Finally, we show that HS messages with check-worthy claims show significantly higher harassment and hate, and that incorporating check-worthiness labels improves LLM-based HS detection up to 0.213 macro-F1 and to 0.154 macro-F1 on average for large models.

Related papers

• Wisdom of the LLM Crowd: A Large Scale Benchmark of Multi-Label U.S. Election-Related Harmful Social Media Content [1.7020765368200692]
  USE24-XD is a large-scale dataset of nearly 100k posts collected from X (formerly Twitter) during the 2024 U.S presidential election cycle.<n>We employ six large language models (LLMs) to systematically annotate posts across five categories: Conspiracy, Sensationalism, Hate Speech, Speculation, and Satire.
  arXiv  Detail & Related papers  (2026-02-12T13:57:12Z)
• Large Language Model Hacking: Quantifying the Hidden Risks of Using LLMs for Text Annotation [66.84286617519258]
  Large language models are transforming social science research by enabling the automation of labor-intensive tasks like data annotation and text analysis.<n>Such variation can introduce systematic biases and random errors, which propagate to downstream analyses and cause Type I (false positive), Type II (false negative), Type S (wrong sign), or Type M (exaggerated effect) errors.<n>We find that intentional LLM hacking is strikingly simple. By replicating 37 data annotation tasks from 21 published social science studies, we show that, with just a handful of prompt paraphrases, virtually anything can be presented as statistically significant.
  arXiv  Detail & Related papers  (2025-09-10T17:58:53Z)
• Beyond Prompt-Induced Lies: Investigating LLM Deception on Benign Prompts [79.1081247754018]
  Large Language Models (LLMs) are widely deployed in reasoning, planning, and decision-making tasks.<n>We propose a framework based on Contact Searching Questions(CSQ) to quantify the likelihood of deception.
  arXiv  Detail & Related papers  (2025-08-08T14:46:35Z)
• Verifying the Verifiers: Unveiling Pitfalls and Potentials in Fact Verifiers [59.168391398830515]
  We evaluate 12 pre-trained LLMs and one specialized fact-verifier, using a collection of examples from 14 fact-checking benchmarks.<n>We highlight the importance of addressing annotation errors and ambiguity in datasets.<n> frontier LLMs with few-shot in-context examples, often overlooked in previous works, achieve top-tier performance.
  arXiv  Detail & Related papers  (2025-06-16T10:32:10Z)
• Generating Grounded Responses to Counter Misinformation via Learning Efficient Fine-Grained Critiques [9.514892000592912]
  MisMitiFact is an efficient framework for generating fact-grounded counter-responses at scale.<n>We develop lightweight, fine-grained critique models trained on data sourced from readily available fact-checking sites.<n>It achieves 5x increase in feedback generation throughput, making it highly suitable for cost-effective, large-scale misinformation mitigation.
  arXiv  Detail & Related papers  (2025-06-06T09:46:09Z)
• Revisiting Backdoor Attacks on LLMs: A Stealthy and Practical Poisoning Framework via Harmless Inputs [54.90315421117162]
  We propose a novel poisoning method via completely harmless data.<n>Inspired by the causal reasoning in auto-regressive LLMs, we aim to establish robust associations between triggers and an affirmative response prefix.<n>We observe an interesting resistance phenomenon where the LLM initially appears to agree but subsequently refuses to answer.
  arXiv  Detail & Related papers  (2025-05-23T08:13:59Z)
• IssueBench: Millions of Realistic Prompts for Measuring Issue Bias in LLM Writing Assistance [49.605835017477034]
  IssueBench is a set of 2.49m realistic English-language prompts to measure issue bias in large language models.<n>Using IssueBench, we show that issue biases are common and persistent in 10 state-of-the-art LLMs.
  arXiv  Detail & Related papers  (2025-02-12T13:37:03Z)
• HateBench: Benchmarking Hate Speech Detectors on LLM-Generated Content and Hate Campaigns [29.913089752247362]
  Large Language Models (LLMs) have raised increasing concerns about their misuse in generating hate speech.<n>We propose HateBench, a framework for benchmarking hate speech detectors on LLM-generated hate speech.<n>Our results show that while detectors are generally effective in identifying LLM-generated hate speech, their performance degrades with newer versions of LLMs.
  arXiv  Detail & Related papers  (2025-01-28T07:00:45Z)
• HiddenGuard: Fine-Grained Safe Generation with Specialized Representation Router [42.222681564769076]
  We introduce HiddenGuard, a novel framework for fine-grained, safe generation in Large Language Models. HiddenGuard incorporates Prism, which operates alongside the LLM to enable real-time, token-level detection and redaction of harmful content. Our experiments demonstrate that HiddenGuard achieves over 90% in F1 score for detecting and redacting harmful content.
  arXiv  Detail & Related papers  (2024-10-03T17:10:41Z)
• CrAM: Credibility-Aware Attention Modification in LLMs for Combating Misinformation in RAG [50.030526904378256]
  Retrieval-Augmented Generation (RAG) can alleviate hallucinations of Large Language Models (LLMs) by referencing external documents.<n>To address this issue, we explore the task of "credibility-aware RAG"<n>We introduce a plug-and-play method named $textbfCr$edibility-aware $textbfA$ttention $textbfM$odification (CrAM)<n>Experiments on Natual Questions and TriviaQA using Llama2-13B, Llama3-8B, and Qwen1.5-7B show that CrAM improves
  arXiv  Detail & Related papers  (2024-06-17T13:01:12Z)
• Don't Go To Extremes: Revealing the Excessive Sensitivity and Calibration Limitations of LLMs in Implicit Hate Speech Detection [29.138463029748547]
  This paper explores the capability of Large Language Models to detect implicit hate speech and express confidence in their responses. Our findings highlight that LLMs exhibit two extremes: (1) LLMs display excessive sensitivity towards groups or topics that may cause fairness issues, resulting in misclassifying benign statements as hate speech.
  arXiv  Detail & Related papers  (2024-02-18T00:04:40Z)
• The Earth is Flat? Unveiling Factual Errors in Large Language Models [89.94270049334479]
  Large Language Models (LLMs) like ChatGPT are in various applications due to their extensive knowledge from pre-training and fine-tuning. Despite this, they are prone to generating factual and commonsense errors, raising concerns in critical areas like healthcare, journalism, and education. We introduce a novel, automatic testing framework, FactChecker, aimed at uncovering factual inaccuracies in LLMs.
  arXiv  Detail & Related papers  (2024-01-01T14:02:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.