MoESD: Mixture of Experts Stable Diffusion to Mitigate Gender Bias

• URL: http://arxiv.org/abs/2407.11002v2
• Date: Thu, 24 Oct 2024 11:28:27 GMT
• Title: MoESD: Mixture of Experts Stable Diffusion to Mitigate Gender Bias
• Authors: Guorun Wang, Lucia Specia,
• Abstract summary: We introduce a Mixture-of-Experts approach to mitigate gender bias in text-to-image models. We show that our approach successfully mitigates gender bias while maintaining image quality.
• Score: 23.10522891268232
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Text-to-image models are known to propagate social biases. For example, when prompted to generate images of people in certain professions, these models tend to systematically generate specific genders or ethnicities. In this paper, we show that this bias is already present in the text encoder of the model and introduce a Mixture-of-Experts approach by identifying text-encoded bias in the latent space and then creating a Bias-Identification Gate mechanism. More specifically, we propose MoESD (Mixture of Experts Stable Diffusion) with BiAs (Bias Adapters) to mitigate gender bias in text-to-image models. We also demonstrate that introducing an arbitrary special token to the prompt is essential during the mitigation process. With experiments focusing on gender bias, we show that our approach successfully mitigates gender bias while maintaining image quality.

Related papers

• GradBias: Unveiling Word Influence on Bias in Text-to-Image Generative Models [75.04426753720553]
  We propose a framework to identify, quantify, and explain biases in an open set setting. This pipeline leverages a Large Language Model (LLM) to propose biases starting from a set of captions. We show two variations of this framework: OpenBias and GradBias.
  arXiv  Detail & Related papers  (2024-08-29T16:51:07Z)
• Gender Bias Evaluation in Text-to-image Generation: A Survey [25.702257177921048]
  We review recent work on gender bias evaluation in text-to-image generation. We focus on the evaluation of recent popular models such as Stable Diffusion and DALL-E 2.
  arXiv  Detail & Related papers  (2024-08-21T06:01:23Z)
• MIST: Mitigating Intersectional Bias with Disentangled Cross-Attention Editing in Text-to-Image Diffusion Models [3.3454373538792552]
  We introduce a method that addresses intersectional bias in diffusion-based text-to-image models by modifying cross-attention maps in a disentangled manner. Our approach utilizes a pre-trained Stable Diffusion model, eliminates the need for an additional set of reference images, and preserves the original quality for unaltered concepts.
  arXiv  Detail & Related papers  (2024-03-28T17:54:38Z)
• The Male CEO and the Female Assistant: Evaluation and Mitigation of Gender Biases in Text-To-Image Generation of Dual Subjects [58.27353205269664]
  We propose the Paired Stereotype Test (PST) framework, which queries T2I models to depict two individuals assigned with male-stereotyped and female-stereotyped social identities. PST queries T2I models to depict two individuals assigned with male-stereotyped and female-stereotyped social identities. Using PST, we evaluate two aspects of gender biases -- the well-known bias in gendered occupation and a novel aspect: bias in organizational power.
  arXiv  Detail & Related papers  (2024-02-16T21:32:27Z)
• VisoGender: A dataset for benchmarking gender bias in image-text pronoun resolution [80.57383975987676]
  VisoGender is a novel dataset for benchmarking gender bias in vision-language models. We focus on occupation-related biases within a hegemonic system of binary gender, inspired by Winograd and Winogender schemas. We benchmark several state-of-the-art vision-language models and find that they demonstrate bias in resolving binary gender in complex scenes.
  arXiv  Detail & Related papers  (2023-06-21T17:59:51Z)
• Balancing the Picture: Debiasing Vision-Language Datasets with Synthetic Contrast Sets [52.77024349608834]
  Vision-language models can perpetuate and amplify societal biases learned during pre-training on uncurated image-text pairs from the internet. COCO Captions is the most commonly used dataset for evaluating bias between background context and the gender of people in-situ. We propose a novel dataset debiasing pipeline to augment the COCO dataset with synthetic, gender-balanced contrast sets.
  arXiv  Detail & Related papers  (2023-05-24T17:59:18Z)
• Auditing Gender Presentation Differences in Text-to-Image Models [54.16959473093973]
  We study how gender is presented differently in text-to-image models. By probing gender indicators in the input text, we quantify the frequency differences of presentation-centric attributes. We propose an automatic method to estimate such differences.
  arXiv  Detail & Related papers  (2023-02-07T18:52:22Z)
• Are Gender-Neutral Queries Really Gender-Neutral? Mitigating Gender Bias in Image Search [8.730027941735804]
  We study a unique gender bias in image search. The search images are often gender-imbalanced for gender-neutral natural language queries. We introduce two novel debiasing approaches.
  arXiv  Detail & Related papers  (2021-09-12T04:47:33Z)
• Understanding and Mitigating Annotation Bias in Facial Expression Recognition [3.325054486984015]
  Most existing works assume that human-generated annotations can be considered gold-standard and unbiased. We focus on facial expression recognition and compare the label biases between lab-controlled and in-the-wild datasets. We propose an AU-Calibrated Facial Expression Recognition framework that utilizes facial action units (AUs) and incorporates the triplet loss into the objective function.
  arXiv  Detail & Related papers  (2021-08-19T05:28:07Z)
• Stereotype and Skew: Quantifying Gender Bias in Pre-trained and Fine-tuned Language Models [5.378664454650768]
  This paper proposes two intuitive metrics, skew and stereotype, that quantify and analyse the gender bias present in contextual language models. We find evidence that gender stereotype correlates approximately negatively with gender skew in out-of-the-box models, suggesting that there is a trade-off between these two forms of bias.
  arXiv  Detail & Related papers  (2021-01-24T10:57:59Z)
• Mitigating Gender Bias in Captioning Systems [56.25457065032423]
  Most captioning models learn gender bias, leading to high gender prediction errors, especially for women. We propose a new Guided Attention Image Captioning model (GAIC) which provides self-guidance on visual attention to encourage the model to capture correct gender visual evidence.
  arXiv  Detail & Related papers  (2020-06-15T12:16:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.