Exploring Gender Bias in Retrieval Models

• URL: http://arxiv.org/abs/2208.01755v1
• Date: Tue, 2 Aug 2022 21:12:05 GMT
• Title: Exploring Gender Bias in Retrieval Models
• Authors: Dhanasekar Sundararaman, Vivek Subramanian
• Abstract summary: Mitigating gender bias in information retrieval is important to avoid propagating stereotypes. We employ a dataset consisting of two components: (1) relevance of a document to a query and (2) "gender" of a document. We show that pre-trained models for IR do not perform well in zero-shot retrieval tasks when full fine-tuning of a large pre-trained BERT encoder is performed. We also illustrate that pre-trained models have gender biases that result in retrieved articles tending to be more often male than female.
• Score: 2.594412743115663
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Biases in culture, gender, ethnicity, etc. have existed for decades and have affected many areas of human social interaction. These biases have been shown to impact machine learning (ML) models, and for natural language processing (NLP), this can have severe consequences for downstream tasks. Mitigating gender bias in information retrieval (IR) is important to avoid propagating stereotypes. In this work, we employ a dataset consisting of two components: (1) relevance of a document to a query and (2) "gender" of a document, in which pronouns are replaced by male, female, and neutral conjugations. We definitively show that pre-trained models for IR do not perform well in zero-shot retrieval tasks when full fine-tuning of a large pre-trained BERT encoder is performed and that lightweight fine-tuning performed with adapter networks improves zero-shot retrieval performance almost by 20% over baseline. We also illustrate that pre-trained models have gender biases that result in retrieved articles tending to be more often male than female. We overcome this by introducing a debiasing technique that penalizes the model when it prefers males over females, resulting in an effective model that retrieves articles in a balanced fashion across genders.

Related papers

• Revealing and Reducing Gender Biases in Vision and Language Assistants (VLAs) [82.57490175399693]
  We study gender bias in 22 popular image-to-text vision-language assistants (VLAs) Our results show that VLAs replicate human biases likely present in the data, such as real-world occupational imbalances. To eliminate the gender bias in these models, we find that finetuning-based debiasing methods achieve the best tradeoff between debiasing and retaining performance on downstream tasks.
  arXiv  Detail & Related papers  (2024-10-25T05:59:44Z)
• The Impact of Debiasing on the Performance of Language Models in Downstream Tasks is Underestimated [70.23064111640132]
  We compare the impact of debiasing on performance across multiple downstream tasks using a wide-range of benchmark datasets. Experiments show that the effects of debiasing are consistently emphunderestimated across all tasks.
  arXiv  Detail & Related papers  (2023-09-16T20:25:34Z)
• The Birth of Bias: A case study on the evolution of gender bias in an English language model [1.6344851071810076]
  We use a relatively small language model, using the LSTM architecture trained on an English Wikipedia corpus. We find that the representation of gender is dynamic and identify different phases during training. We show that gender information is represented increasingly locally in the input embeddings of the model.
  arXiv  Detail & Related papers  (2022-07-21T00:59:04Z)
• Towards Understanding Gender-Seniority Compound Bias in Natural Language Generation [64.65911758042914]
  We investigate how seniority impacts the degree of gender bias exhibited in pretrained neural generation models. Our results show that GPT-2 amplifies bias by considering women as junior and men as senior more often than the ground truth in both domains. These results suggest that NLP applications built using GPT-2 may harm women in professional capacities.
  arXiv  Detail & Related papers  (2022-05-19T20:05:02Z)
• Improving Gender Fairness of Pre-Trained Language Models without Catastrophic Forgetting [88.83117372793737]
  Forgetting information in the original training data may damage the model's downstream performance by a large margin. We propose GEnder Equality Prompt (GEEP) to improve gender fairness of pre-trained models with less forgetting.
  arXiv  Detail & Related papers  (2021-10-11T15:52:16Z)
• Are Gender-Neutral Queries Really Gender-Neutral? Mitigating Gender Bias in Image Search [8.730027941735804]
  We study a unique gender bias in image search. The search images are often gender-imbalanced for gender-neutral natural language queries. We introduce two novel debiasing approaches.
  arXiv  Detail & Related papers  (2021-09-12T04:47:33Z)
• Collecting a Large-Scale Gender Bias Dataset for Coreference Resolution and Machine Translation [10.542861450223128]
  We find grammatical patterns indicating stereotypical and non-stereotypical gender-role assignments in corpora from three domains. We manually verify the quality of our corpus and use it to evaluate gender bias in various coreference resolution and machine translation models.
  arXiv  Detail & Related papers  (2021-09-08T18:14:11Z)
• Mitigating Gender Bias in Captioning Systems [56.25457065032423]
  Most captioning models learn gender bias, leading to high gender prediction errors, especially for women. We propose a new Guided Attention Image Captioning model (GAIC) which provides self-guidance on visual attention to encourage the model to capture correct gender visual evidence.
  arXiv  Detail & Related papers  (2020-06-15T12:16:19Z)
• Multi-Dimensional Gender Bias Classification [67.65551687580552]
  Machine learning models can inadvertently learn socially undesirable patterns when training on gender biased text. We propose a general framework that decomposes gender bias in text along several pragmatic and semantic dimensions. Using this fine-grained framework, we automatically annotate eight large scale datasets with gender information.
  arXiv  Detail & Related papers  (2020-05-01T21:23:20Z)
• Do Neural Ranking Models Intensify Gender Bias? [13.37092521347171]
  We first provide a bias measurement framework which includes two metrics to quantify the degree of the unbalanced presence of gender-related concepts in a given IR model's ranking list. Applying these queries to the MS MARCO Passage retrieval collection, we then measure the gender bias of a BM25 model and several recent neural ranking models. Results show that while all models are strongly biased toward male, the neural models, and in particular the ones based on contextualized embedding models, significantly intensify gender bias.
  arXiv  Detail & Related papers  (2020-05-01T13:31:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.