Evaluating Bias and Fairness in Gender-Neutral Pretrained Vision-and-Language Models

• URL: http://arxiv.org/abs/2310.17530v1
• Date: Thu, 26 Oct 2023 16:19:19 GMT
• Title: Evaluating Bias and Fairness in Gender-Neutral Pretrained Vision-and-Language Models
• Authors: Laura Cabello, Emanuele Bugliarello, Stephanie Brandl, Desmond Elliott
• Abstract summary: We quantify bias amplification in pretraining and after fine-tuning on three families of vision-and-language models. Overall, we find that bias amplification in pretraining and after fine-tuning are independent.
• Score: 23.65626682262062
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Pretrained machine learning models are known to perpetuate and even amplify existing biases in data, which can result in unfair outcomes that ultimately impact user experience. Therefore, it is crucial to understand the mechanisms behind those prejudicial biases to ensure that model performance does not result in discriminatory behaviour toward certain groups or populations. In this work, we define gender bias as our case study. We quantify bias amplification in pretraining and after fine-tuning on three families of vision-and-language models. We investigate the connection, if any, between the two learning stages, and evaluate how bias amplification reflects on model performance. Overall, we find that bias amplification in pretraining and after fine-tuning are independent. We then examine the effect of continued pretraining on gender-neutral data, finding that this reduces group disparities, i.e., promotes fairness, on VQAv2 and retrieval tasks without significantly compromising task performance.

Related papers

• Revealing and Reducing Gender Biases in Vision and Language Assistants (VLAs) [82.57490175399693]
  We study gender bias in 22 popular image-to-text vision-language assistants (VLAs) Our results show that VLAs replicate human biases likely present in the data, such as real-world occupational imbalances. To eliminate the gender bias in these models, we find that finetuning-based debiasing methods achieve the best tradeoff between debiasing and retaining performance on downstream tasks.
  arXiv  Detail & Related papers  (2024-10-25T05:59:44Z)
• Less can be more: representational vs. stereotypical gender bias in facial expression recognition [3.9698529891342207]
  Machine learning models can inherit biases from their training data, leading to discriminatory or inaccurate predictions. This paper investigates the propagation of demographic biases from datasets into machine learning models. We focus on the gender demographic component, analyzing two types of bias: representational and stereotypical.
  arXiv  Detail & Related papers  (2024-06-25T09:26:49Z)
• Identifying and examining machine learning biases on Adult dataset [0.7856362837294112]
  This research delves into the reduction of machine learning model bias through Ensemble Learning. Our rigorous methodology comprehensively assesses bias across various categorical variables, ultimately revealing a pronounced gender attribute bias. This study underscores ethical considerations and advocates the implementation of hybrid models for a data-driven society marked by inclusivity and impartiality.
  arXiv  Detail & Related papers  (2023-10-13T19:41:47Z)
• Language Models Get a Gender Makeover: Mitigating Gender Bias with Few-Shot Data Interventions [50.67412723291881]
  Societal biases present in pre-trained large language models are a critical issue. We propose data intervention strategies as a powerful yet simple technique to reduce gender bias in pre-trained models.
  arXiv  Detail & Related papers  (2023-06-07T16:50:03Z)
• Deep Learning on a Healthy Data Diet: Finding Important Examples for Fairness [15.210232622716129]
  Data-driven predictive solutions predominant in commercial applications tend to suffer from biases and stereotypes. Data augmentation reduces gender bias by adding counterfactual examples to the training dataset. We show that some of the examples in the augmented dataset can be not important or even harmful for fairness.
  arXiv  Detail & Related papers  (2022-11-20T22:42:30Z)
• Increasing Fairness in Predictions Using Bias Parity Score Based Loss Function Regularization [0.8594140167290099]
  We introduce a family of fairness enhancing regularization components that we use in conjunction with the traditional binary-cross-entropy based accuracy loss. We deploy them in the context of a recidivism prediction task as well as on a census-based adult income dataset.
  arXiv  Detail & Related papers  (2021-11-05T17:42:33Z)
• Improving Gender Fairness of Pre-Trained Language Models without Catastrophic Forgetting [88.83117372793737]
  Forgetting information in the original training data may damage the model's downstream performance by a large margin. We propose GEnder Equality Prompt (GEEP) to improve gender fairness of pre-trained models with less forgetting.
  arXiv  Detail & Related papers  (2021-10-11T15:52:16Z)
• Balancing out Bias: Achieving Fairness Through Training Reweighting [58.201275105195485]
  Bias in natural language processing arises from models learning characteristics of the author such as gender and race. Existing methods for mitigating and measuring bias do not directly account for correlations between author demographics and linguistic variables. This paper introduces a very simple but highly effective method for countering bias using instance reweighting.
  arXiv  Detail & Related papers  (2021-09-16T23:40:28Z)
• Unravelling the Effect of Image Distortions for Biased Prediction of Pre-trained Face Recognition Models [86.79402670904338]
  We evaluate the performance of four state-of-the-art deep face recognition models in the presence of image distortions. We have observed that image distortions have a relationship with the performance gap of the model across different subgroups.
  arXiv  Detail & Related papers  (2021-08-14T16:49:05Z)
• Impact of Gender Debiased Word Embeddings in Language Modeling [0.0]
  Gender, race and social biases have been detected as evident examples of unfairness in applications of Natural Language Processing. Recent studies have shown that the human-generated data used in training is an apparent factor of getting biases. Current algorithms have also been proven to amplify biases from data.
  arXiv  Detail & Related papers  (2021-05-03T14:45:10Z)
• Mitigating Gender Bias Amplification in Distribution by Posterior Regularization [75.3529537096899]
  We investigate the gender bias amplification issue from the distribution perspective. We propose a bias mitigation approach based on posterior regularization. Our study sheds the light on understanding the bias amplification.
  arXiv  Detail & Related papers  (2020-05-13T11:07:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.