IYKYK (But AI Doesn't): Automated Content Moderation Does Not Capture Communities' Heterogeneous Attitudes Towards Reclaimed Language

• URL: http://arxiv.org/abs/2604.16654v2
• Date: Tue, 21 Apr 2026 17:02:41 GMT
• Title: IYKYK (But AI Doesn't): Automated Content Moderation Does Not Capture Communities' Heterogeneous Attitudes Towards Reclaimed Language
• Abstract summary: We use quantitative and qualitative methods to examine attitudes of social media users in LGBTQIA+, Black, and women communities around reclaimed slurs.<n>We observe low inter-annotator agreement, indicating substantial disagreement among in-group annotators.<n>We find poor alignment between annotator judgments and automated hate speech assessments produced by Perspective API.
• Score: 45.4201325387611
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reclaimed slur usage is a common and meaningful practice online for many marginalized communities. It serves as a source of solidarity, identity, and shared experience. However, contemporary automated and AI-based moderation tools for online content largely fail to distinguish between reclaimed and hateful uses of slurs, resulting in the suppression of marginalized voices. In this work, we use quantitative and qualitative methods to examine the attitudes of social media users in LGBTQIA+, Black, and women communities around reclaimed slurs targeting our focus groups including the f-word, n-word, and b-word. With social media users from these communities, we collect and analyze an annotated online slur usage corpus. The corpus includes annotators' perceptions of whether an online text containing a slur should be flagged as hate speech, as well as contextual features of the slur usage. Across all communities and annotation questions, we observe low inter-annotator agreement, indicating substantial disagreement among in-group annotators. This is compounded by the fact that, absent clear contextual signals of identity and intent, even in-group members may disagree on how to interpret reclaimed slur usage online. Semi-structured interviews with annotators suggest that differences in lived experience and personal history contribute to this variation as well. We find poor alignment between annotator judgments and automated hate speech assessments produced by Perspective API. We further observe that certain features of a text such as whether the slur usage was derogatory and if the slur was targeted at oneself are more associated with whether annotators report the text as hate speech. Together, these findings highlight the inherent subjectivity and contextual nature of how marginalized communities interpret slurs online.

Related papers

• Explain the Flag: Contextualizing Hate Speech Beyond Censorship [2.796818629124347]
  We present a hybrid approach that combines Large Language Models (LLMs) with three newly created and curated vocabularies to detect and explain hate speech in English, French, and Greek.<n>Our system captures both inherently derogatory expressions tied to identity characteristics and direct group-targeted content through two complementary pipelines.<n>The outputs are fused into grounded explanations that clarify why content is flagged.
  arXiv  Detail & Related papers  (2026-04-16T13:06:28Z)
• SLAyiNG: Towards Queer Language Processing [44.4984082814346]
  SLAyiNG is the first dataset containing annotated queer slang derived from subtitles, social media posts, and podcasts.<n>We describe our data curation process, including the collection of slang terms and definitions, scraping sources for examples that reflect usage of these terms.<n>As preliminary results, we calculate inter-annotator agreement for human annotators and OpenAI's model o3-mini.
  arXiv  Detail & Related papers  (2025-09-22T07:41:45Z)
• Identity-related Speech Suppression in Generative AI Content Moderation [2.2020053359163305]
  We find that identity-related speech is more likely to be incorrectly suppressed than other speech.<n>We find that reasons for incorrect flagging behavior vary by identity based on stereotypes and text associations.<n>As generative AI systems are increasingly used for creative work, we urge further attention to how this may impact the creation of identity-related content.
  arXiv  Detail & Related papers  (2024-09-09T14:34:51Z)
• Analyzing Norm Violations in Live-Stream Chat [49.120561596550395]
  We study the first NLP study dedicated to detecting norm violations in conversations on live-streaming platforms. We define norm violation categories in live-stream chats and annotate 4,583 moderated comments from Twitch. Our results show that appropriate contextual information can boost moderation performance by 35%.
  arXiv  Detail & Related papers  (2023-05-18T05:58:27Z)
• CoSyn: Detecting Implicit Hate Speech in Online Conversations Using a Context Synergized Hyperbolic Network [52.85130555886915]
  CoSyn is a context-synergized neural network that explicitly incorporates user- and conversational context for detecting implicit hate speech in online conversations. We show that CoSyn outperforms all our baselines in detecting implicit hate speech with absolute improvements in the range of 1.24% - 57.8%.
  arXiv  Detail & Related papers  (2023-03-02T17:30:43Z)
• Countering Malicious Content Moderation Evasion in Online Social Networks: Simulation and Detection of Word Camouflage [64.78260098263489]
  Twisting and camouflaging keywords are among the most used techniques to evade platform content moderation systems. This article contributes significantly to countering malicious information by developing multilingual tools to simulate and detect new methods of evasion of content.
  arXiv  Detail & Related papers  (2022-12-27T16:08:49Z)
• Enriching Abusive Language Detection with Community Context [0.3708656266586145]
  Use of pejorative expressions can be benign or actively empowering. Models for abuse detection misclassify these expressions as derogatory, inadvertently censor productive conversations held by marginalized groups. Our paper highlights how community context can improve classification outcomes in abusive language detection.
  arXiv  Detail & Related papers  (2022-06-16T20:54:02Z)
• Hate Speech and Counter Speech Detection: Conversational Context Does Matter [7.333666276087548]
  This paper investigates the role of conversational context in the annotation and detection of online hate and counter speech. We created a context-aware dataset for a 3-way classification task on Reddit comments: hate speech, counter speech, or neutral.
  arXiv  Detail & Related papers  (2022-06-13T19:05:44Z)
• Beyond Plain Toxic: Detection of Inappropriate Statements on Flammable Topics for the Russian Language [76.58220021791955]
  We present two text collections labelled according to binary notion of inapropriateness and a multinomial notion of sensitive topic. To objectivise the notion of inappropriateness, we define it in a data-driven way though crowdsourcing.
  arXiv  Detail & Related papers  (2022-03-04T15:59:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.