Mapping Toxic Comments Across Demographics: A Dataset from German Public Broadcasting

• URL: http://arxiv.org/abs/2508.21084v1
• Date: Tue, 26 Aug 2025 16:51:15 GMT
• Title: Mapping Toxic Comments Across Demographics: A Dataset from German Public Broadcasting
• Abstract summary: This dataset includes 3,024 human-annotated and 30,024 anonymized comments from Instagram, TikTok, and YouTube.<n>To ensure relevance, comments were consolidated using predefined toxic keywords, resulting in 16.7% labeled as problematic.<n>The dataset reveals age-based differences in toxic speech patterns, with younger users favoring expressive language and older users more often engaging in disinformation and devaluation.
• Score: 0.8466401378239364
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: A lack of demographic context in existing toxic speech datasets limits our understanding of how different age groups communicate online. In collaboration with funk, a German public service content network, this research introduces the first large-scale German dataset annotated for toxicity and enriched with platform-provided age estimates. The dataset includes 3,024 human-annotated and 30,024 LLM-annotated anonymized comments from Instagram, TikTok, and YouTube. To ensure relevance, comments were consolidated using predefined toxic keywords, resulting in 16.7\% labeled as problematic. The annotation pipeline combined human expertise with state-of-the-art language models, identifying key categories such as insults, disinformation, and criticism of broadcasting fees. The dataset reveals age-based differences in toxic speech patterns, with younger users favoring expressive language and older users more often engaging in disinformation and devaluation. This resource provides new opportunities for studying linguistic variation across demographics and supports the development of more equitable and age-aware content moderation systems.

Related papers

• Assessing the Level of Toxicity Against Distinct Groups in Bangla Social Media Comments: A Comprehensive Investigation [0.0]
  This study focuses on identifying toxic comments in the Bengali language targeting three specific groups: transgender people, indigenous people, and migrant people. The methodology involves creating a dataset, manual annotation, and employing pre-trained transformer models like Bangla-BERT, bangla-bert-base, distil-BERT, and Bert-base-multilingual-cased for classification. The experimental findings reveal that Bangla-BERT surpasses alternative models, achieving an F1-score of 0.8903.
  arXiv  Detail & Related papers  (2024-09-25T17:48:59Z)
• The Evolution of Language in Social Media Comments [37.69303106863453]
  This study investigates the linguistic characteristics of user comments over 34 years, focusing on their complexity and temporal shifts. We utilize a dataset of approximately 300 million English comments from eight diverse platforms and topics. Our findings reveal consistent patterns of complexity across social media platforms and topics, characterized by a nearly universal reduction in text length, diminished lexical richness, but decreased repetitiveness.
  arXiv  Detail & Related papers  (2024-06-17T12:03:30Z)
• Developing Linguistic Patterns to Mitigate Inherent Human Bias in Offensive Language Detection [1.6574413179773761]
  We propose a linguistic data augmentation approach to reduce bias in labeling processes. This approach has the potential to improve offensive language classification tasks across multiple languages.
  arXiv  Detail & Related papers  (2023-12-04T10:20:36Z)
• Models See Hallucinations: Evaluating the Factuality in Video Captioning [57.85548187177109]
  We conduct a human evaluation of the factuality in video captioning and collect two annotated factuality datasets. We find that 57.0% of the model-generated sentences have factual errors, indicating it is a severe problem in this field. We propose a weakly-supervised, model-based factuality metric FactVC, which outperforms previous metrics on factuality evaluation of video captioning.
  arXiv  Detail & Related papers  (2023-03-06T08:32:50Z)
• Countering Malicious Content Moderation Evasion in Online Social Networks: Simulation and Detection of Word Camouflage [64.78260098263489]
  Twisting and camouflaging keywords are among the most used techniques to evade platform content moderation systems. This article contributes significantly to countering malicious information by developing multilingual tools to simulate and detect new methods of evasion of content.
  arXiv  Detail & Related papers  (2022-12-27T16:08:49Z)
• ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection [33.715318646717385]
  ToxiGen is a large-scale dataset of 274k toxic and benign statements about 13 minority groups. Controlling machine generation in this way allows ToxiGen to cover implicitly toxic text at a larger scale. We find that 94.5% of toxic examples are labeled as hate speech by human annotators.
  arXiv  Detail & Related papers  (2022-03-17T17:57:56Z)
• A New Generation of Perspective API: Efficient Multilingual Character-level Transformers [66.9176610388952]
  We present the fundamentals behind the next version of the Perspective API from Google Jigsaw. At the heart of the approach is a single multilingual token-free Charformer model. We demonstrate that by forgoing static vocabularies, we gain flexibility across a variety of settings.
  arXiv  Detail & Related papers  (2022-02-22T20:55:31Z)
• Addressing the Challenges of Cross-Lingual Hate Speech Detection [115.1352779982269]
  In this paper we focus on cross-lingual transfer learning to support hate speech detection in low-resource languages. We leverage cross-lingual word embeddings to train our neural network systems on the source language and apply it to the target language. We investigate the issue of label imbalance of hate speech datasets, since the high ratio of non-hate examples compared to hate examples often leads to low model performance.
  arXiv  Detail & Related papers  (2022-01-15T20:48:14Z)
• Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detection [75.54119209776894]
  We investigate the effect of annotator identities (who) and beliefs (why) on toxic language annotations. We consider posts with three characteristics: anti-Black language, African American English dialect, and vulgarity. Our results show strong associations between annotator identity and beliefs and their ratings of toxicity.
  arXiv  Detail & Related papers  (2021-11-15T18:58:20Z)
• Toxic Language Detection in Social Media for Brazilian Portuguese: New Dataset and Multilingual Analysis [4.251937086394346]
  State-of-the-art BERT models were able to achieve 76% macro-F1 score using monolingual data in the binary case. We show that large-scale monolingual data is still needed to create more accurate models.
  arXiv  Detail & Related papers  (2020-10-09T13:05:19Z)
• Multilingual Twitter Corpus and Baselines for Evaluating Demographic Bias in Hate Speech Recognition [46.57105755981092]
  We publish a multilingual Twitter corpus for the task of hate speech detection. The corpus covers five languages: English, Italian, Polish, Portuguese and Spanish. We evaluate the inferred demographic labels with a crowdsourcing platform.
  arXiv  Detail & Related papers  (2020-02-24T16:45:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.