Do Neural Ranking Models Intensify Gender Bias?

• URL: http://arxiv.org/abs/2005.00372v3
• Date: Mon, 15 Jun 2020 13:11:59 GMT
• Title: Do Neural Ranking Models Intensify Gender Bias?
• Authors: Navid Rekabsaz, Markus Schedl
• Abstract summary: We first provide a bias measurement framework which includes two metrics to quantify the degree of the unbalanced presence of gender-related concepts in a given IR model's ranking list. Applying these queries to the MS MARCO Passage retrieval collection, we then measure the gender bias of a BM25 model and several recent neural ranking models. Results show that while all models are strongly biased toward male, the neural models, and in particular the ones based on contextualized embedding models, significantly intensify gender bias.
• Score: 13.37092521347171
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Concerns regarding the footprint of societal biases in information retrieval (IR) systems have been raised in several previous studies. In this work, we examine various recent IR models from the perspective of the degree of gender bias in their retrieval results. To this end, we first provide a bias measurement framework which includes two metrics to quantify the degree of the unbalanced presence of gender-related concepts in a given IR model's ranking list. To examine IR models by means of the framework, we create a dataset of non-gendered queries, selected by human annotators. Applying these queries to the MS MARCO Passage retrieval collection, we then measure the gender bias of a BM25 model and several recent neural ranking models. The results show that while all models are strongly biased toward male, the neural models, and in particular the ones based on contextualized embedding models, significantly intensify gender bias. Our experiments also show an overall increase in the gender bias of neural models when they exploit transfer learning, namely when they use (already biased) pre-trained embeddings.

Related papers

• Dataset Distribution Impacts Model Fairness: Single vs. Multi-Task Learning [2.9530211066840417]
  We evaluate the performance of skin lesion classification using ResNet-based CNNs. We present a linear programming method for generating datasets with varying patient sex and class labels.
  arXiv  Detail & Related papers  (2024-07-24T15:23:26Z)
• AI Gender Bias, Disparities, and Fairness: Does Training Data Matter? [3.509963616428399]
  This study delves into the pervasive issue of gender issues in artificial intelligence (AI) It analyzes more than 1000 human-graded student responses from male and female participants across six assessment items. Results indicate that scoring accuracy for mixed-trained models shows an insignificant difference from either male- or female-trained models.
  arXiv  Detail & Related papers  (2023-12-17T22:37:06Z)
• Fast Model Debias with Machine Unlearning [54.32026474971696]
  Deep neural networks might behave in a biased manner in many real-world scenarios. Existing debiasing methods suffer from high costs in bias labeling or model re-training. We propose a fast model debiasing framework (FMD) which offers an efficient approach to identify, evaluate and remove biases.
  arXiv  Detail & Related papers  (2023-10-19T08:10:57Z)
• Studying the Effects of Sex-related Differences on Brain Age Prediction using brain MR Imaging [0.3958317527488534]
  We study biases related to sex when developing a machine learning model based on brain magnetic resonance images (MRI) We investigate the effects of sex by performing brain age prediction considering different experimental designs. We found disparities in the performance of brain age prediction models when trained on distinct sex subgroups and datasets.
  arXiv  Detail & Related papers  (2023-10-17T20:55:53Z)
• Identifying and examining machine learning biases on Adult dataset [0.7856362837294112]
  This research delves into the reduction of machine learning model bias through Ensemble Learning. Our rigorous methodology comprehensively assesses bias across various categorical variables, ultimately revealing a pronounced gender attribute bias. This study underscores ethical considerations and advocates the implementation of hybrid models for a data-driven society marked by inclusivity and impartiality.
  arXiv  Detail & Related papers  (2023-10-13T19:41:47Z)
• Exploring Gender Bias in Retrieval Models [2.594412743115663]
  Mitigating gender bias in information retrieval is important to avoid propagating stereotypes. We employ a dataset consisting of two components: (1) relevance of a document to a query and (2) "gender" of a document. We show that pre-trained models for IR do not perform well in zero-shot retrieval tasks when full fine-tuning of a large pre-trained BERT encoder is performed. We also illustrate that pre-trained models have gender biases that result in retrieved articles tending to be more often male than female.
  arXiv  Detail & Related papers  (2022-08-02T21:12:05Z)
• Pseudo Bias-Balanced Learning for Debiased Chest X-ray Classification [57.53567756716656]
  We study the problem of developing debiased chest X-ray diagnosis models without knowing exactly the bias labels. We propose a novel algorithm, pseudo bias-balanced learning, which first captures and predicts per-sample bias labels. Our proposed method achieved consistent improvements over other state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-03-18T11:02:18Z)
• Are Commercial Face Detection Models as Biased as Academic Models? [64.71318433419636]
  We compare academic and commercial face detection systems, specifically examining robustness to noise. We find that state-of-the-art academic face detection models exhibit demographic disparities in their noise robustness. We conclude that commercial models are always as biased or more biased than an academic model.
  arXiv  Detail & Related papers  (2022-01-25T02:21:42Z)
• Mitigating Gender Bias in Captioning Systems [56.25457065032423]
  Most captioning models learn gender bias, leading to high gender prediction errors, especially for women. We propose a new Guided Attention Image Captioning model (GAIC) which provides self-guidance on visual attention to encourage the model to capture correct gender visual evidence.
  arXiv  Detail & Related papers  (2020-06-15T12:16:19Z)
• Multi-Dimensional Gender Bias Classification [67.65551687580552]
  Machine learning models can inadvertently learn socially undesirable patterns when training on gender biased text. We propose a general framework that decomposes gender bias in text along several pragmatic and semantic dimensions. Using this fine-grained framework, we automatically annotate eight large scale datasets with gender information.
  arXiv  Detail & Related papers  (2020-05-01T21:23:20Z)
• InsideBias: Measuring Bias in Deep Networks and Application to Face Gender Biometrics [73.85525896663371]
  This work explores the biases in learning processes based on deep neural network architectures. We employ two gender detection models based on popular deep neural networks. We propose InsideBias, a novel method to detect biased models.
  arXiv  Detail & Related papers  (2020-04-14T15:20:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.