Stable Diffusion Exposed: Gender Bias from Prompt to Image

• URL: http://arxiv.org/abs/2312.03027v2
• Date: Sun, 11 Aug 2024 08:10:32 GMT
• Title: Stable Diffusion Exposed: Gender Bias from Prompt to Image
• Authors: Yankun Wu, Yuta Nakashima, Noa Garcia,
• Abstract summary: This paper introduces an evaluation protocol that analyzes the impact of gender indicators at every step of the generation process on Stable Diffusion images. Our findings include the existence of differences in the depiction of objects, such as instruments tailored for specific genders, and shifts in overall layouts.
• Score: 25.702257177921048
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Several studies have raised awareness about social biases in image generative models, demonstrating their predisposition towards stereotypes and imbalances. This paper contributes to this growing body of research by introducing an evaluation protocol that analyzes the impact of gender indicators at every step of the generation process on Stable Diffusion images. Leveraging insights from prior work, we explore how gender indicators not only affect gender presentation but also the representation of objects and layouts within the generated images. Our findings include the existence of differences in the depiction of objects, such as instruments tailored for specific genders, and shifts in overall layouts. We also reveal that neutral prompts tend to produce images more aligned with masculine prompts than their feminine counterparts. We further explore where bias originates through representational disparities and how it manifests in the images via prompt-image dependencies, and provide recommendations for developers and users to mitigate potential bias in image generation.

Related papers

• MoESD: Mixture of Experts Stable Diffusion to Mitigate Gender Bias [23.10522891268232]
  We introduce a Mixture-of-Experts approach to mitigate gender bias in text-to-image models. We show that our approach successfully mitigates gender bias while maintaining image quality.
  arXiv  Detail & Related papers  (2024-06-25T14:59:31Z)
• The Male CEO and the Female Assistant: Evaluation and Mitigation of Gender Biases in Text-To-Image Generation of Dual Subjects [58.27353205269664]
  We propose the Paired Stereotype Test (PST) framework, which queries T2I models to depict two individuals assigned with male-stereotyped and female-stereotyped social identities. PST queries T2I models to depict two individuals assigned with male-stereotyped and female-stereotyped social identities. Using PST, we evaluate two aspects of gender biases -- the well-known bias in gendered occupation and a novel aspect: bias in organizational power.
  arXiv  Detail & Related papers  (2024-02-16T21:32:27Z)
• VisoGender: A dataset for benchmarking gender bias in image-text pronoun resolution [80.57383975987676]
  VisoGender is a novel dataset for benchmarking gender bias in vision-language models. We focus on occupation-related biases within a hegemonic system of binary gender, inspired by Winograd and Winogender schemas. We benchmark several state-of-the-art vision-language models and find that they demonstrate bias in resolving binary gender in complex scenes.
  arXiv  Detail & Related papers  (2023-06-21T17:59:51Z)
• Stereotypes and Smut: The (Mis)representation of Non-cisgender Identities by Text-to-Image Models [6.92043136971035]
  We investigate how multimodal models handle diverse gender identities. We find certain non-cisgender identities are consistently (mis)represented as less human, more stereotyped and more sexualised. These improvements could pave the way for a future where change is led by the affected community.
  arXiv  Detail & Related papers  (2023-05-26T16:28:49Z)
• Gender Biases in Automatic Evaluation Metrics for Image Captioning [87.15170977240643]
  We conduct a systematic study of gender biases in model-based evaluation metrics for image captioning tasks. We demonstrate the negative consequences of using these biased metrics, including the inability to differentiate between biased and unbiased generations. We present a simple and effective way to mitigate the metric bias without hurting the correlations with human judgments.
  arXiv  Detail & Related papers  (2023-05-24T04:27:40Z)
• Stable Bias: Analyzing Societal Representations in Diffusion Models [72.27121528451528]
  We propose a new method for exploring the social biases in Text-to-Image (TTI) systems. Our approach relies on characterizing the variation in generated images triggered by enumerating gender and ethnicity markers in the prompts. We leverage this method to analyze images generated by 3 popular TTI systems and find that while all of their outputs show correlations with US labor demographics, they also consistently under-represent marginalized identities to different extents.
  arXiv  Detail & Related papers  (2023-03-20T19:32:49Z)
• Auditing Gender Presentation Differences in Text-to-Image Models [54.16959473093973]
  We study how gender is presented differently in text-to-image models. By probing gender indicators in the input text, we quantify the frequency differences of presentation-centric attributes. We propose an automatic method to estimate such differences.
  arXiv  Detail & Related papers  (2023-02-07T18:52:22Z)
• How well can Text-to-Image Generative Models understand Ethical Natural Language Interventions? [67.97752431429865]
  We study the effect on the diversity of the generated images when adding ethical intervention. Preliminary studies indicate that a large change in the model predictions is triggered by certain phrases such as 'irrespective of gender'
  arXiv  Detail & Related papers  (2022-10-27T07:32:39Z)
• Gender Artifacts in Visual Datasets [34.74191865400569]
  We investigate what $textitgender artifacts$ exist within large-scale visual datasets. We find that gender artifacts are ubiquitous in the COCO and OpenImages datasets. We claim that attempts to remove gender artifacts from such datasets are largely infeasible.
  arXiv  Detail & Related papers  (2022-06-18T12:09:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.