Identification of Bias Against People with Disabilities in Sentiment Analysis and Toxicity Detection Models

• URL: http://arxiv.org/abs/2111.13259v1
• Date: Thu, 25 Nov 2021 21:44:18 GMT
• Title: Identification of Bias Against People with Disabilities in Sentiment Analysis and Toxicity Detection Models
• Authors: Pranav Narayanan Venkit, Shomir Wilson
• Abstract summary: We present the Bias Identification Test in Sentiments (BITS), a corpus of 1,126 sentences designed to probe sentiment analysis models for biases in disability. Results show that all exhibit strong negative biases on sentences that mention disability.
• Score: 0.5758109624133713
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Sociodemographic biases are a common problem for natural language processing, affecting the fairness and integrity of its applications. Within sentiment analysis, these biases may undermine sentiment predictions for texts that mention personal attributes that unbiased human readers would consider neutral. Such discrimination can have great consequences in the applications of sentiment analysis both in the public and private sectors. For example, incorrect inferences in applications like online abuse and opinion analysis in social media platforms can lead to unwanted ramifications, such as wrongful censoring, towards certain populations. In this paper, we address the discrimination against people with disabilities, PWD, done by sentiment analysis and toxicity classification models. We provide an examination of sentiment and toxicity analysis models to understand in detail how they discriminate PWD. We present the Bias Identification Test in Sentiments (BITS), a corpus of 1,126 sentences designed to probe sentiment analysis models for biases in disability. We use this corpus to demonstrate statistically significant biases in four widely used sentiment analysis tools (TextBlob, VADER, Google Cloud Natural Language API and DistilBERT) and two toxicity analysis models trained to predict toxic comments on Jigsaw challenges (Toxic comment classification and Unintended Bias in Toxic comments). The results show that all exhibit strong negative biases on sentences that mention disability. We publicly release BITS Corpus for others to identify potential biases against disability in any sentiment analysis tools and also to update the corpus to be used as a test for other sociodemographic variables as well.

Related papers

• You Shall Know a Tool by the Traces it Leaves: The Predictability of Sentiment Analysis Tools [74.98850427240464]
  We show that sentiment analysis tools disagree on the same dataset. We show that the sentiment tool used for sentiment annotation can even be predicted from its outcome.
  arXiv  Detail & Related papers  (2024-10-18T17:27:38Z)
• Lexicon-Based Sentiment Analysis on Text Polarities with Evaluation of Classification Models [1.342834401139078]
  This work uses a lexicon-based method to perform sentiment analysis and shows an evaluation of classification models trained over textual data. The lexicon-based methods identify the intensity of emotion and subjectivity at word levels. This work is based on a multi-class problem of text being labeled as positive, negative, or neutral.
  arXiv  Detail & Related papers  (2024-09-19T15:31:12Z)
• The Devil is in the Neurons: Interpreting and Mitigating Social Biases in Pre-trained Language Models [78.69526166193236]
  Pre-trained Language models (PLMs) have been acknowledged to contain harmful information, such as social biases. We propose sc Social Bias Neurons to accurately pinpoint units (i.e., neurons) in a language model that can be attributed to undesirable behavior, such as social bias. As measured by prior metrics from StereoSet, our model achieves a higher degree of fairness while maintaining language modeling ability with low cost.
  arXiv  Detail & Related papers  (2024-06-14T15:41:06Z)
• Diagnosing and Debiasing Corpus-Based Political Bias and Insults in GPT2 [0.0]
  Training large language models (LLMs) on extensive, unfiltered corpora sourced from the internet is a common and advantageous practice. Recent research shows that generative pretrained transformer (GPT) language models can recognize their own biases and detect toxicity in generated content. This study investigates the efficacy of the diagnosing-debiasing approach in mitigating two additional types of biases: insults and political bias.
  arXiv  Detail & Related papers  (2023-11-17T01:20:08Z)
• Decoding Susceptibility: Modeling Misbelief to Misinformation Through a Computational Approach [61.04606493712002]
  Susceptibility to misinformation describes the degree of belief in unverifiable claims that is not observable. Existing susceptibility studies heavily rely on self-reported beliefs. We propose a computational approach to model users' latent susceptibility levels.
  arXiv  Detail & Related papers  (2023-11-16T07:22:56Z)
• Automated Ableism: An Exploration of Explicit Disability Biases in Sentiment and Toxicity Analysis Models [5.611973529240434]
  We analyze sentiment analysis and toxicity detection models to detect the presence of explicit bias against people with disability (PWD) We employ the bias identification framework of Perturbation Sensitivity Analysis to examine conversations related to PWD on social media platforms, specifically Twitter and Reddit. We then create the textitBias Identification Test in Sentiment (BITS) corpus to quantify explicit disability bias in any sentiment analysis and toxicity detection models.
  arXiv  Detail & Related papers  (2023-07-18T12:45:54Z)
• Toxicity Detection with Generative Prompt-based Inference [3.9741109244650823]
  It is a long-known risk that language models (LMs), once trained on corpus containing undesirable content, have the power to manifest biases and toxicity. In this work, we explore the generative variant of zero-shot prompt-based toxicity detection with comprehensive trials on prompt engineering.
  arXiv  Detail & Related papers  (2022-05-24T22:44:43Z)
• Anatomizing Bias in Facial Analysis [86.79402670904338]
  Existing facial analysis systems have been shown to yield biased results against certain demographic subgroups. It has become imperative to ensure that these systems do not discriminate based on gender, identity, or skin tone of individuals. This has led to research in the identification and mitigation of bias in AI systems.
  arXiv  Detail & Related papers  (2021-12-13T09:51:13Z)
• Mitigating Biases in Toxic Language Detection through Invariant Rationalization [70.36701068616367]
  biases toward some attributes, including gender, race, and dialect, exist in most training datasets for toxicity detection. We propose to use invariant rationalization (InvRat), a game-theoretic framework consisting of a rationale generator and a predictor, to rule out the spurious correlation of certain syntactic patterns. Our method yields lower false positive rate in both lexical and dialectal attributes than previous debiasing methods.
  arXiv  Detail & Related papers  (2021-06-14T08:49:52Z)
• Challenges in Automated Debiasing for Toxic Language Detection [81.04406231100323]
  Biased associations have been a challenge in the development of classifiers for detecting toxic language. We investigate recently introduced debiasing methods for text classification datasets and models, as applied to toxic language detection. Our focus is on lexical (e.g., swear words, slurs, identity mentions) and dialectal markers (specifically African American English)
  arXiv  Detail & Related papers  (2021-01-29T22:03:17Z)
• Social Biases in NLP Models as Barriers for Persons with Disabilities [13.579848462349192]
  We present evidence of undesirable biases towards mentions of disability in two different English language models: toxicity prediction and sentiment analysis. Next, we demonstrate that the neural embeddings that are the critical first step in most NLP pipelines similarly contain undesirable biases towards mentions of disability. We end by highlighting topical biases in the discourse about disability which may contribute to the observed model biases.
  arXiv  Detail & Related papers  (2020-05-02T12:16:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.