The Tail Wagging the Dog: Dataset Construction Biases of Social Bias Benchmarks

• URL: http://arxiv.org/abs/2210.10040v2
• Date: Fri, 16 Jun 2023 18:35:13 GMT
• Title: The Tail Wagging the Dog: Dataset Construction Biases of Social Bias Benchmarks
• Authors: Nikil Roashan Selvam, Sunipa Dev, Daniel Khashabi, Tushar Khot, Kai-Wei Chang
• Abstract summary: We compare social biases with non-social biases stemming from choices made during dataset construction that might not even be discernible to the human eye. We observe that these shallow modifications have a surprising effect on the resulting degree of bias across various models.
• Score: 75.58692290694452
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: How reliably can we trust the scores obtained from social bias benchmarks as faithful indicators of problematic social biases in a given language model? In this work, we study this question by contrasting social biases with non-social biases stemming from choices made during dataset construction that might not even be discernible to the human eye. To do so, we empirically simulate various alternative constructions for a given benchmark based on innocuous modifications (such as paraphrasing or random-sampling) that maintain the essence of their social bias. On two well-known social bias benchmarks (Winogender and BiasNLI) we observe that these shallow modifications have a surprising effect on the resulting degree of bias across various models. We hope these troubling observations motivate more robust measures of social biases.

Related papers

• VLBiasBench: A Comprehensive Benchmark for Evaluating Bias in Large Vision-Language Model [72.13121434085116]
  VLBiasBench is a benchmark aimed at evaluating biases in Large Vision-Language Models (LVLMs) We construct a dataset encompassing nine distinct categories of social biases, including age, disability status, gender, nationality, physical appearance, race, religion, profession, social economic status and two intersectional bias categories (race x gender, and race x social economic status) We conduct extensive evaluations on 15 open-source models as well as one advanced closed-source model, providing some new insights into the biases revealing from these models.
  arXiv  Detail & Related papers  (2024-06-20T10:56:59Z)
• The Devil is in the Neurons: Interpreting and Mitigating Social Biases in Pre-trained Language Models [78.69526166193236]
  Pre-trained Language models (PLMs) have been acknowledged to contain harmful information, such as social biases. We propose sc Social Bias Neurons to accurately pinpoint units (i.e., neurons) in a language model that can be attributed to undesirable behavior, such as social bias. As measured by prior metrics from StereoSet, our model achieves a higher degree of fairness while maintaining language modeling ability with low cost.
  arXiv  Detail & Related papers  (2024-06-14T15:41:06Z)
• Semantic Properties of cosine based bias scores for word embeddings [52.13994416317707]
  We propose requirements for bias scores to be considered meaningful for quantifying biases. We analyze cosine based scores from the literature with regard to these requirements. We underline these findings with experiments to show that the bias scores' limitations have an impact in the application case.
  arXiv  Detail & Related papers  (2024-01-27T20:31:10Z)
• SocialStigmaQA: A Benchmark to Uncover Stigma Amplification in Generative Language Models [8.211129045180636]
  We introduce a benchmark meant to capture the amplification of social bias, via stigmas, in generative language models. Our benchmark, SocialStigmaQA, contains roughly 10K prompts, with a variety of prompt styles, carefully constructed to test for both social bias and model robustness. We find that the proportion of socially biased output ranges from 45% to 59% across a variety of decoding strategies and prompting styles.
  arXiv  Detail & Related papers  (2023-12-12T18:27:44Z)
• Social Bias Probing: Fairness Benchmarking for Language Models [38.180696489079985]
  This paper proposes a novel framework for probing language models for social biases by assessing disparate treatment. We curate SOFA, a large-scale benchmark designed to address the limitations of existing fairness collections. Comparing our methodology with existing benchmarks, we reveal that biases within language models are more nuanced than acknowledged.
  arXiv  Detail & Related papers  (2023-11-15T16:35:59Z)
• Probing Intersectional Biases in Vision-Language Models with Counterfactual Examples [5.870913541790421]
  We employ text-to-image diffusion models to produce counterfactual examples for probing intserctional social biases at scale. Our approach utilizes Stable Diffusion with cross attention control to produce sets of counterfactual image-text pairs. We conduct extensive experiments using our generated dataset which reveal the intersectional social biases present in state-of-the-art VLMs.
  arXiv  Detail & Related papers  (2023-10-04T17:25:10Z)
• The SAME score: Improved cosine based bias score for word embeddings [63.24247894974291]
  We provide a bias definition based on the ideas from the literature and derive novel requirements for bias scores. We propose a new bias score, SAME, to address the shortcomings of existing bias scores and show empirically that SAME is better suited to quantify biases in word embeddings.
  arXiv  Detail & Related papers  (2022-03-28T09:28:13Z)
• BBQ: A Hand-Built Bias Benchmark for Question Answering [25.108222728383236]
  It is well documented that NLP models learn social biases present in the world, but little work has been done to show how these biases manifest in actual model outputs for applied tasks like question answering (QA) We introduce the Bias Benchmark for QA (BBQ), a dataset consisting of question-sets constructed by the authors that highlight textitattested social biases against people belonging to protected classes along nine different social dimensions relevant for U.S. English-speaking contexts. We find that models strongly rely on stereotypes when the context is ambiguous, meaning that the model's outputs consistently reproduce harmful biases in this setting
  arXiv  Detail & Related papers  (2021-10-15T16:43:46Z)
• Towards Understanding and Mitigating Social Biases in Language Models [107.82654101403264]
  Large-scale pretrained language models (LMs) can be potentially dangerous in manifesting undesirable representational biases. We propose steps towards mitigating social biases during text generation. Our empirical results and human evaluation demonstrate effectiveness in mitigating bias while retaining crucial contextual information.
  arXiv  Detail & Related papers  (2021-06-24T17:52:43Z)
• Argument from Old Man's View: Assessing Social Bias in Argumentation [20.65183968971417]
  Social bias in language poses a problem with ethical impact for many NLP applications. Recent research has shown that machine learning models trained on respective data may not only adopt, but even amplify the bias. We study the existence of social biases in large English debate portals.
  arXiv  Detail & Related papers  (2020-11-24T10:39:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.