Bias in Text Embedding Models

• URL: http://arxiv.org/abs/2406.12138v1
• Date: Mon, 17 Jun 2024 22:58:36 GMT
• Title: Bias in Text Embedding Models
• Authors: Vasyl Rakivnenko, Nestor Maslej, Jessica Cervi, Volodymyr Zhukov,
• Abstract summary: This paper examines the degree to which a selection of popular text embedding models are biased, particularly along gendered dimensions. The analysis reveals that text embedding models are prone to gendered biases but in varying ways.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Text embedding is becoming an increasingly popular AI methodology, especially among businesses, yet the potential of text embedding models to be biased is not well understood. This paper examines the degree to which a selection of popular text embedding models are biased, particularly along gendered dimensions. More specifically, this paper studies the degree to which these models associate a list of given professions with gendered terms. The analysis reveals that text embedding models are prone to gendered biases but in varying ways. Although there are certain inter-model commonalities, for instance, greater association of professions like nurse, homemaker, and socialite with female identifiers, and greater association of professions like CEO, manager, and boss with male identifiers, not all models make the same gendered associations for each occupation. Furthermore, the magnitude and directionality of bias can also vary on a model-by-model basis and depend on the particular words models are prompted with. This paper demonstrates that gender bias afflicts text embedding models and suggests that businesses using this technology need to be mindful of the specific dimensions of this problem.

Related papers

• Gender Bias in Decision-Making with Large Language Models: A Study of Relationship Conflicts [15.676219253088211]
  We study gender equity within large language models (LLMs) through a decision-making lens. We explore nine relationship configurations through name pairs across three name lists (men, women, neutral)
  arXiv  Detail & Related papers  (2024-10-14T20:50:11Z)
• GenderBias-\emph{VL}: Benchmarking Gender Bias in Vision Language Models via Counterfactual Probing [72.0343083866144]
  This paper introduces the GenderBias-emphVL benchmark to evaluate occupation-related gender bias in Large Vision-Language Models. Using our benchmark, we extensively evaluate 15 commonly used open-source LVLMs and state-of-the-art commercial APIs. Our findings reveal widespread gender biases in existing LVLMs.
  arXiv  Detail & Related papers  (2024-06-30T05:55:15Z)
• Generalizing Fairness to Generative Language Models via Reformulation of Non-discrimination Criteria [4.738231680800414]
  This paper studies how to uncover and quantify the presence of gender biases in generative language models. We derive generative AI analogues of three well-known non-discrimination criteria from classification, namely independence, separation and sufficiency. Our results address the presence of occupational gender bias within such conversational language models.
  arXiv  Detail & Related papers  (2024-03-13T14:19:08Z)
• Multilingual Text-to-Image Generation Magnifies Gender Stereotypes and Prompt Engineering May Not Help You [64.74707085021858]
  We show that multilingual models suffer from significant gender biases just as monolingual models do. We propose a novel benchmark, MAGBIG, intended to foster research on gender bias in multilingual models. Our results show that not only do models exhibit strong gender biases but they also behave differently across languages.
  arXiv  Detail & Related papers  (2024-01-29T12:02:28Z)
• VisoGender: A dataset for benchmarking gender bias in image-text pronoun resolution [80.57383975987676]
  VisoGender is a novel dataset for benchmarking gender bias in vision-language models. We focus on occupation-related biases within a hegemonic system of binary gender, inspired by Winograd and Winogender schemas. We benchmark several state-of-the-art vision-language models and find that they demonstrate bias in resolving binary gender in complex scenes.
  arXiv  Detail & Related papers  (2023-06-21T17:59:51Z)
• Efficient Gender Debiasing of Pre-trained Indic Language Models [0.0]
  The gender bias present in the data on which language models are pre-trained gets reflected in the systems that use these models. In our paper, we measure gender bias associated with occupations in Hindi language models. Our results reflect that the bias is reduced post-introduction of our proposed mitigation techniques.
  arXiv  Detail & Related papers  (2022-09-08T09:15:58Z)
• Are Commercial Face Detection Models as Biased as Academic Models? [64.71318433419636]
  We compare academic and commercial face detection systems, specifically examining robustness to noise. We find that state-of-the-art academic face detection models exhibit demographic disparities in their noise robustness. We conclude that commercial models are always as biased or more biased than an academic model.
  arXiv  Detail & Related papers  (2022-01-25T02:21:42Z)
• Collecting a Large-Scale Gender Bias Dataset for Coreference Resolution and Machine Translation [10.542861450223128]
  We find grammatical patterns indicating stereotypical and non-stereotypical gender-role assignments in corpora from three domains. We manually verify the quality of our corpus and use it to evaluate gender bias in various coreference resolution and machine translation models.
  arXiv  Detail & Related papers  (2021-09-08T18:14:11Z)
• How True is GPT-2? An Empirical Analysis of Intersectional Occupational Biases [50.591267188664666]
  Downstream applications are at risk of inheriting biases contained in natural language models. We analyze the occupational biases of a popular generative language model, GPT-2. For a given job, GPT-2 reflects the societal skew of gender and ethnicity in the US, and in some cases, pulls the distribution towards gender parity.
  arXiv  Detail & Related papers  (2021-02-08T11:10:27Z)
• Multi-Dimensional Gender Bias Classification [67.65551687580552]
  Machine learning models can inadvertently learn socially undesirable patterns when training on gender biased text. We propose a general framework that decomposes gender bias in text along several pragmatic and semantic dimensions. Using this fine-grained framework, we automatically annotate eight large scale datasets with gender information.
  arXiv  Detail & Related papers  (2020-05-01T21:23:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.