A Low-Cost Human-in-the-Loop Investigation of Toxicity on GitHub at Scale

• URL: http://arxiv.org/abs/2607.25946v2
• Date: Wed, 29 Jul 2026 23:37:54 GMT
• Title: A Low-Cost Human-in-the-Loop Investigation of Toxicity on GitHub at Scale
• Abstract summary: Toxic interactions in open source discussions can alienate contributors and threaten project sustainability.<n>We present a human-in-the-loop (HITL) annotation methodology that makes large-scale, domain-calibrated toxicity labeling practical.
• Score: 6.53794042872403
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Toxic interactions in open source discussions can alienate contributors and threaten project sustainability, yet prior empirical studies of GitHub toxicity have been limited in scale, raising questions about their generalizability. Scaling up is difficult because toxicity on GitHub is often implicit and context-dependent, making both fully manual annotation and LLM-based labeling unreliable. We present a human-in-the-loop (HITL) annotation methodology that makes large-scale, domain-calibrated toxicity labeling practical. A single call to a small, local LLM produces both a toxicity prediction and a set of interpretable event category scores. A lightweight Random Forest validator then uses those scores to flag the small subset of conversations most likely to be mislabeled, directing human review only where it is needed. The validator outperforms confidence-based and multi-LLM baselines while adding low annotation cost. We apply this pipeline to over 124,000 GitHub issue and pull request conversations. Using the resulting dataset, we evaluate key findings from prior small-scale research, confirming some and qualifying others, and present new insights into the prevalence, characteristics, and dynamics of toxicity across diverse open source projects.

Related papers

• Toxicity Ahead: Forecasting Conversational Derailment on GitHub [9.862809084820304]
  Toxic interactions in Open Source Software (OSS) communities reduce contributor engagement and threaten project sustainability.<n>To support more scalable approaches, we curate a dataset of 159 derailed toxic threads and 207 non-toxic threads from GitHub discussions.<n>We present a novel Large Language Model (LLM)-based framework for predicting conversational derailment on GitHub using a two-step prompting pipeline.
  arXiv  Detail & Related papers  (2025-12-17T02:45:12Z)
• Consistency Is the Key: Detecting Hallucinations in LLM Generated Text By Checking Inconsistencies About Key Facts [21.081815261690444]
  Large language models (LLMs) often hallucinate and generate text that is factually incorrect and not grounded in real-world knowledge.<n>This poses serious risks in domains like healthcare, finance, and customer support.<n>We introduce CONFACTCHECK, an efficient detection approach that does not leverage any external knowledge base.
  arXiv  Detail & Related papers  (2025-11-15T14:33:02Z)
• Key-Augmented Neural Triggers for Knowledge Sharing [3.8718804267599416]
  Key-Augmented Neural Triggers embeds knowledge anchors into both training and inference.<n>It reduces fragmentation and grounding inference in localized context.<n>It is well-suited for scalable, low-latency, on-premise deployments.
  arXiv  Detail & Related papers  (2025-08-05T11:40:56Z)
• Can Reasoning Help Large Language Models Capture Human Annotator Disagreement? [84.32752330104775]
  Variation in human annotation (i.e., disagreements) is common in NLP.<n>We evaluate the influence of different reasoning settings on Large Language Model disagreement modeling.<n>Surprisingly, our results show that RLVR-style reasoning degrades performance in disagreement modeling.
  arXiv  Detail & Related papers  (2025-06-24T09:49:26Z)
• Search Arena: Analyzing Search-Augmented LLMs [61.28673331156436]
  We introduce Search Arena, a crowd-sourced, large-scale, human-preference dataset of over 24,000 paired multi-turn user interactions.<n>The dataset spans diverse intents and languages, and contains full system traces with around 12,000 human preference votes.<n>Our analysis reveals that user preferences are influenced by the number of citations, even when the cited content does not directly support the attributed claims.
  arXiv  Detail & Related papers  (2025-06-05T17:59:26Z)
• Understanding and Predicting Derailment in Toxic Conversations on GitHub [6.343946534579351]
  This study aims to understand and predict conversational derailment leading to toxicity on GitHub.<n>Based on this dataset, we identify unique characteristics of toxic conversations and derailment points.<n>We propose a proactive moderation approach to automatically detect and address potentially harmful conversations before escalation.
  arXiv  Detail & Related papers  (2025-03-04T02:01:37Z)
• The Landscape of Toxicity: An Empirical Investigation of Toxicity on GitHub [3.0586855806896054]
  profanity is the most frequent toxicity on GitHub, followed by trolling and insults.<n>Corporate-sponsored projects are less toxic, but gaming projects are seven times more toxic than non-gaming ones.<n> OSS contributors who have authored toxic comments in the past are significantly more likely to repeat such behavior.
  arXiv  Detail & Related papers  (2025-02-12T09:24:59Z)
• Exploring ChatGPT for Toxicity Detection in GitHub [5.003898791753481]
  The prevalence of negative discourse, often manifested as toxic comments, poses significant challenges to developer well-being and productivity. To identify such negativity in project communications, automated toxicity detection models are necessary. To train these models effectively, we need large software engineering-specific toxicity datasets.
  arXiv  Detail & Related papers  (2023-12-20T15:23:00Z)
• CoAnnotating: Uncertainty-Guided Work Allocation between Human and Large Language Models for Data Annotation [94.59630161324013]
  We propose CoAnnotating, a novel paradigm for Human-LLM co-annotation of unstructured texts at scale. Our empirical study shows CoAnnotating to be an effective means to allocate work from results on different datasets, with up to 21% performance improvement over random baseline.
  arXiv  Detail & Related papers  (2023-10-24T08:56:49Z)
• FactCHD: Benchmarking Fact-Conflicting Hallucination Detection [64.4610684475899]
  FactCHD is a benchmark designed for the detection of fact-conflicting hallucinations from LLMs. FactCHD features a diverse dataset that spans various factuality patterns, including vanilla, multi-hop, comparison, and set operation. We introduce Truth-Triangulator that synthesizes reflective considerations by tool-enhanced ChatGPT and LoRA-tuning based on Llama2.
  arXiv  Detail & Related papers  (2023-10-18T16:27:49Z)
• A New Benchmark and Reverse Validation Method for Passage-level Hallucination Detection [63.56136319976554]
  Large Language Models (LLMs) generate hallucinations, which can cause significant damage when deployed for mission-critical tasks. We propose a self-check approach based on reverse validation to detect factual errors automatically in a zero-resource fashion. We empirically evaluate our method and existing zero-resource detection methods on two datasets.
  arXiv  Detail & Related papers  (2023-10-10T10:14:59Z)
• Bring Your Own Data! Self-Supervised Evaluation for Large Language Models [52.15056231665816]
  We propose a framework for self-supervised evaluation of Large Language Models (LLMs) We demonstrate self-supervised evaluation strategies for measuring closed-book knowledge, toxicity, and long-range context dependence. We find strong correlations between self-supervised and human-supervised evaluations.
  arXiv  Detail & Related papers  (2023-06-23T17:59:09Z)
• On the Blind Spots of Model-Based Evaluation Metrics for Text Generation [79.01422521024834]
  We explore a useful but often neglected methodology for robustness analysis of text generation evaluation metrics. We design and synthesize a wide range of potential errors and check whether they result in a commensurate drop in the metric scores. Our experiments reveal interesting insensitivities, biases, or even loopholes in existing metrics.
  arXiv  Detail & Related papers  (2022-12-20T06:24:25Z)
• A Token-level Reference-free Hallucination Detection Benchmark for Free-form Text Generation [50.55448707570669]
  We propose a novel token-level, reference-free hallucination detection task and an associated annotated dataset named HaDes. To create this dataset, we first perturb a large number of text segments extracted from English language Wikipedia, and then verify these with crowd-sourced annotations.
  arXiv  Detail & Related papers  (2021-04-18T04:09:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.