Detecting Unintended Social Bias in Toxic Language Datasets

• URL: http://arxiv.org/abs/2210.11762v1
• Date: Fri, 21 Oct 2022 06:50:12 GMT
• Title: Detecting Unintended Social Bias in Toxic Language Datasets
• Authors: Nihar Sahoo, Himanshu Gupta, Pushpak Bhattacharyya
• Abstract summary: This paper introduces a new dataset ToxicBias curated from the existing dataset of Kaggle competition named "Jigsaw Unintended Bias in Toxicity Classification" The dataset contains instances annotated for five different bias categories, viz., gender, race/ethnicity, religion, political, and LGBTQ. We train transformer-based models using our curated datasets and report baseline performance for bias identification, target generation, and bias implications.
• Score: 32.724030288421474
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With the rise of online hate speech, automatic detection of Hate Speech, Offensive texts as a natural language processing task is getting popular. However, very little research has been done to detect unintended social bias from these toxic language datasets. This paper introduces a new dataset ToxicBias curated from the existing dataset of Kaggle competition named "Jigsaw Unintended Bias in Toxicity Classification". We aim to detect social biases, their categories, and targeted groups. The dataset contains instances annotated for five different bias categories, viz., gender, race/ethnicity, religion, political, and LGBTQ. We train transformer-based models using our curated datasets and report baseline performance for bias identification, target generation, and bias implications. Model biases and their mitigation are also discussed in detail. Our study motivates a systematic extraction of social bias data from toxic language datasets. All the codes and dataset used for experiments in this work are publicly available

Related papers

• On the Role of Speech Data in Reducing Toxicity Detection Bias [22.44133159647888]
  We produce a set of high-quality group annotations for the multilingual MuTox dataset. We then leverage these annotations to systematically compare speech- and text-based toxicity classifiers. Our findings indicate that access to speech data during inference supports reduced bias against group mentions.
  arXiv  Detail & Related papers  (2024-11-12T19:26:43Z)
• Spoken Stereoset: On Evaluating Social Bias Toward Speaker in Speech Large Language Models [50.40276881893513]
  This study introduces Spoken Stereoset, a dataset specifically designed to evaluate social biases in Speech Large Language Models (SLLMs) By examining how different models respond to speech from diverse demographic groups, we aim to identify these biases. The findings indicate that while most models show minimal bias, some still exhibit slightly stereotypical or anti-stereotypical tendencies.
  arXiv  Detail & Related papers  (2024-08-14T16:55:06Z)
• The Devil is in the Neurons: Interpreting and Mitigating Social Biases in Pre-trained Language Models [78.69526166193236]
  Pre-trained Language models (PLMs) have been acknowledged to contain harmful information, such as social biases. We propose sc Social Bias Neurons to accurately pinpoint units (i.e., neurons) in a language model that can be attributed to undesirable behavior, such as social bias. As measured by prior metrics from StereoSet, our model achieves a higher degree of fairness while maintaining language modeling ability with low cost.
  arXiv  Detail & Related papers  (2024-06-14T15:41:06Z)
• The Impact of Debiasing on the Performance of Language Models in Downstream Tasks is Underestimated [70.23064111640132]
  We compare the impact of debiasing on performance across multiple downstream tasks using a wide-range of benchmark datasets. Experiments show that the effects of debiasing are consistently emphunderestimated across all tasks.
  arXiv  Detail & Related papers  (2023-09-16T20:25:34Z)
• CBBQ: A Chinese Bias Benchmark Dataset Curated with Human-AI Collaboration for Large Language Models [52.25049362267279]
  We present a Chinese Bias Benchmark dataset that consists of over 100K questions jointly constructed by human experts and generative language models. The testing instances in the dataset are automatically derived from 3K+ high-quality templates manually authored with stringent quality control. Extensive experiments demonstrate the effectiveness of the dataset in detecting model bias, with all 10 publicly available Chinese large language models exhibiting strong bias in certain categories.
  arXiv  Detail & Related papers  (2023-06-28T14:14:44Z)
• Exposing Bias in Online Communities through Large-Scale Language Models [3.04585143845864]
  This work uses the flaw of bias in language models to explore the biases of six different online communities. The bias of the resulting models is evaluated by prompting the models with different demographics and comparing the sentiment and toxicity values of these generations. This work not only affirms how easily bias is absorbed from training data but also presents a scalable method to identify and compare the bias of different datasets or communities.
  arXiv  Detail & Related papers  (2023-06-04T08:09:26Z)
• Challenges in Measuring Bias via Open-Ended Language Generation [1.5552869983952944]
  We analyze how specific choices of prompt sets, metrics, automatic tools and sampling strategies affect bias results. We provide recommendations for reporting biases in open-ended language generation.
  arXiv  Detail & Related papers  (2022-05-23T19:57:15Z)
• Mitigating Biases in Toxic Language Detection through Invariant Rationalization [70.36701068616367]
  biases toward some attributes, including gender, race, and dialect, exist in most training datasets for toxicity detection. We propose to use invariant rationalization (InvRat), a game-theoretic framework consisting of a rationale generator and a predictor, to rule out the spurious correlation of certain syntactic patterns. Our method yields lower false positive rate in both lexical and dialectal attributes than previous debiasing methods.
  arXiv  Detail & Related papers  (2021-06-14T08:49:52Z)
• Challenges in Automated Debiasing for Toxic Language Detection [81.04406231100323]
  Biased associations have been a challenge in the development of classifiers for detecting toxic language. We investigate recently introduced debiasing methods for text classification datasets and models, as applied to toxic language detection. Our focus is on lexical (e.g., swear words, slurs, identity mentions) and dialectal markers (specifically African American English)
  arXiv  Detail & Related papers  (2021-01-29T22:03:17Z)
• Hate Speech Detection and Racial Bias Mitigation in Social Media based on BERT model [1.9336815376402716]
  We introduce a transfer learning approach for hate speech detection based on an existing pre-trained language model called BERT. We evaluate the proposed model on two publicly available datasets annotated for racism, sexism, hate or offensive content on Twitter.
  arXiv  Detail & Related papers  (2020-08-14T16:47:25Z)
• Discovering and Categorising Language Biases in Reddit [5.670038395203354]
  This paper proposes a data-driven approach to automatically discover language biases encoded in the vocabulary of online discourse communities on Reddit. We use word embeddings to transform text into high-dimensional dense vectors and capture semantic relations between words. We successfully discover gender bias, religion bias, and ethnic bias in different Reddit communities.
  arXiv  Detail & Related papers  (2020-08-06T16:42:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.