The Tail Wagging the Dog: Dataset Construction Biases of Social Bias Benchmarks

• URL: http://arxiv.org/abs/2210.10040v2
• Date: Fri, 16 Jun 2023 18:35:13 GMT
• Title: The Tail Wagging the Dog: Dataset Construction Biases of Social Bias Benchmarks
• Authors: Nikil Roashan Selvam, Sunipa Dev, Daniel Khashabi, Tushar Khot, Kai-Wei Chang
• Abstract summary: We compare social biases with non-social biases stemming from choices made during dataset construction that might not even be discernible to the human eye. We observe that these shallow modifications have a surprising effect on the resulting degree of bias across various models.
• Score: 75.58692290694452
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: How reliably can we trust the scores obtained from social bias benchmarks as faithful indicators of problematic social biases in a given language model? In this work, we study this question by contrasting social biases with non-social biases stemming from choices made during dataset construction that might not even be discernible to the human eye. To do so, we empirically simulate various alternative constructions for a given benchmark based on innocuous modifications (such as paraphrasing or random-sampling) that maintain the essence of their social bias. On two well-known social bias benchmarks (Winogender and BiasNLI) we observe that these shallow modifications have a surprising effect on the resulting degree of bias across various models. We hope these troubling observations motivate more robust measures of social biases.

Related papers

• VLBiasBench: A Comprehensive Benchmark for Evaluating Bias in Large Vision-Language Model [72.13121434085116]
  VLBiasBench is a benchmark aimed at evaluating biases in Large Vision-Language Models (LVLMs) We construct a dataset encompassing nine distinct categories of social biases, including age, disability status, gender, nationality, physical appearance, race, religion, profession, social economic status and two intersectional bias categories (race x gender, and race x social economic status) We conduct extensive evaluations on 15 open-source models as well as one advanced closed-source model, providing some new insights into the biases revealing from these models.
  arXiv  Detail & Related papers  (2024-06-20T10:56:59Z)
• The Devil is in the Neurons: Interpreting and Mitigating Social Biases in Pre-trained Language Models [78.69526166193236]
  Pre-trained Language models (PLMs) have been acknowledged to contain harmful information, such as social biases. We propose sc Social Bias Neurons to accurately pinpoint units (i.e., neurons) in a language model that can be attributed to undesirable behavior, such as social bias. As measured by prior metrics from StereoSet, our model achieves a higher degree of fairness while maintaining language modeling ability with low cost.
  arXiv  Detail & Related papers  (2024-06-14T15:41:06Z)
• Quantifying Bias in Text-to-Image Generative Models [49.60774626839712]
  Bias in text-to-image (T2I) models can propagate unfair social representations and may be used to aggressively market ideas or push controversial agendas. Existing T2I model bias evaluation methods only focus on social biases. We propose an evaluation methodology to quantify general biases in T2I generative models, without any preconceived notions.
  arXiv  Detail & Related papers  (2023-12-20T14:26:54Z)
• SocialStigmaQA: A Benchmark to Uncover Stigma Amplification in Generative Language Models [8.211129045180636]
  We introduce a benchmark meant to capture the amplification of social bias, via stigmas, in generative language models. Our benchmark, SocialStigmaQA, contains roughly 10K prompts, with a variety of prompt styles, carefully constructed to test for both social bias and model robustness. We find that the proportion of socially biased output ranges from 45% to 59% across a variety of decoding strategies and prompting styles.
  arXiv  Detail & Related papers  (2023-12-12T18:27:44Z)
• Social Bias Probing: Fairness Benchmarking for Language Models [38.180696489079985]
  This paper proposes a novel framework for probing language models for social biases by assessing disparate treatment. We curate SoFa, a large-scale benchmark designed to address the limitations of existing fairness collections. We show that biases within language models are more nuanced than acknowledged, indicating a broader scope of encoded biases than previously recognized.
  arXiv  Detail & Related papers  (2023-11-15T16:35:59Z)
• Probing Intersectional Biases in Vision-Language Models with Counterfactual Examples [5.870913541790421]
  We employ text-to-image diffusion models to produce counterfactual examples for probing intserctional social biases at scale. Our approach utilizes Stable Diffusion with cross attention control to produce sets of counterfactual image-text pairs. We conduct extensive experiments using our generated dataset which reveal the intersectional social biases present in state-of-the-art VLMs.
  arXiv  Detail & Related papers  (2023-10-04T17:25:10Z)
• The SAME score: Improved cosine based bias score for word embeddings [49.75878234192369]
  We introduce SAME, a novel bias score for semantic bias in embeddings. We show that SAME is capable of measuring semantic bias and identify potential causes for social bias in downstream tasks.
  arXiv  Detail & Related papers  (2022-03-28T09:28:13Z)
• Towards Understanding and Mitigating Social Biases in Language Models [107.82654101403264]
  Large-scale pretrained language models (LMs) can be potentially dangerous in manifesting undesirable representational biases. We propose steps towards mitigating social biases during text generation. Our empirical results and human evaluation demonstrate effectiveness in mitigating bias while retaining crucial contextual information.
  arXiv  Detail & Related papers  (2021-06-24T17:52:43Z)
• Argument from Old Man's View: Assessing Social Bias in Argumentation [20.65183968971417]
  Social bias in language poses a problem with ethical impact for many NLP applications. Recent research has shown that machine learning models trained on respective data may not only adopt, but even amplify the bias. We study the existence of social biases in large English debate portals.
  arXiv  Detail & Related papers  (2020-11-24T10:39:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.