Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scale

• URL: http://arxiv.org/abs/2211.03759v2
• Date: Wed, 7 Jun 2023 16:36:13 GMT
• Title: Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scale
• Authors: Federico Bianchi, Pratyusha Kalluri, Esin Durmus, Faisal Ladhak, Myra Cheng, Debora Nozza, Tatsunori Hashimoto, Dan Jurafsky, James Zou, Aylin Caliskan
• Abstract summary: We investigate the potential for machine learning models to amplify dangerous and complex stereotypes. We find a broad range of ordinary prompts produce stereotypes, including prompts simply mentioning traits, descriptors, occupations, or objects.
• Score: 61.555788332182395
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Machine learning models that convert user-written text descriptions into images are now widely available online and used by millions of users to generate millions of images a day. We investigate the potential for these models to amplify dangerous and complex stereotypes. We find a broad range of ordinary prompts produce stereotypes, including prompts simply mentioning traits, descriptors, occupations, or objects. For example, we find cases of prompting for basic traits or social roles resulting in images reinforcing whiteness as ideal, prompting for occupations resulting in amplification of racial and gender disparities, and prompting for objects resulting in reification of American norms. Stereotypes are present regardless of whether prompts explicitly mention identity and demographic language or avoid such language. Moreover, stereotypes persist despite mitigation strategies; neither user attempts to counter stereotypes by requesting images with specific counter-stereotypes nor institutional attempts to add system ``guardrails'' have prevented the perpetuation of stereotypes. Our analysis justifies concerns regarding the impacts of today's models, presenting striking exemplars, and connecting these findings with deep insights into harms drawn from social scientific and humanist disciplines. This work contributes to the effort to shed light on the uniquely complex biases in language-vision models and demonstrates the ways that the mass deployment of text-to-image generation models results in mass dissemination of stereotypes and resulting harms.

Related papers

• Gender Bias in Instruction-Guided Speech Synthesis Models [55.2480439325792]
  This study investigates the potential gender bias in how models interpret occupation-related prompts. We explore whether these models exhibit tendencies to amplify gender stereotypes when interpreting such prompts. Our experimental results reveal the model's tendency to exhibit gender bias for certain occupations.
  arXiv  Detail & Related papers  (2025-02-08T17:38:24Z)
• Biased or Flawed? Mitigating Stereotypes in Generative Language Models by Addressing Task-Specific Flaws [12.559028963968247]
  generative language models often reflect and amplify societal biases in their outputs. We propose a targeted stereotype mitigation framework that implicitly mitigates observed stereotypes in generative models. We reduce stereotypical outputs by over 60% across multiple dimensions.
  arXiv  Detail & Related papers  (2024-12-16T03:29:08Z)
• Spoken Stereoset: On Evaluating Social Bias Toward Speaker in Speech Large Language Models [50.40276881893513]
  This study introduces Spoken Stereoset, a dataset specifically designed to evaluate social biases in Speech Large Language Models (SLLMs) By examining how different models respond to speech from diverse demographic groups, we aim to identify these biases. The findings indicate that while most models show minimal bias, some still exhibit slightly stereotypical or anti-stereotypical tendencies.
  arXiv  Detail & Related papers  (2024-08-14T16:55:06Z)
• Who is better at math, Jenny or Jingzhen? Uncovering Stereotypes in Large Language Models [9.734705470760511]
  We use GlobalBias to study a broad set of stereotypes from around the world. We generate character profiles based on given names and evaluate the prevalence of stereotypes in model outputs.
  arXiv  Detail & Related papers  (2024-07-09T14:52:52Z)
• Stereotype Detection in LLMs: A Multiclass, Explainable, and Benchmark-Driven Approach [4.908389661988191]
  This paper introduces the Multi-Grain Stereotype (MGS) dataset, consisting of 51,867 instances across gender, race, profession, religion, and other stereotypes. We evaluate various machine learning approaches to establish baselines and fine-tune language models of different architectures and sizes. We employ explainable AI (XAI) tools, including SHAP, LIME, and BertViz, to assess whether the model's learned patterns align with human intuitions about stereotypes.
  arXiv  Detail & Related papers  (2024-04-02T09:31:32Z)
• Language Agents for Detecting Implicit Stereotypes in Text-to-image Models at Scale [45.64096601242646]
  We introduce a novel agent architecture tailored for stereotype detection in text-to-image models. We build the stereotype-relevant benchmark based on multiple open-text datasets. We find that these models often display serious stereotypes when it comes to certain prompts about personal characteristics.
  arXiv  Detail & Related papers  (2023-10-18T08:16:29Z)
• A Prompt Array Keeps the Bias Away: Debiasing Vision-Language Models with Adversarial Learning [55.96577490779591]
  Vision-language models can encode societal biases and stereotypes. There are challenges to measuring and mitigating these multimodal harms. We investigate bias measures and apply ranking metrics for image-text representations.
  arXiv  Detail & Related papers  (2022-03-22T17:59:04Z)
• Towards Understanding and Mitigating Social Biases in Language Models [107.82654101403264]
  Large-scale pretrained language models (LMs) can be potentially dangerous in manifesting undesirable representational biases. We propose steps towards mitigating social biases during text generation. Our empirical results and human evaluation demonstrate effectiveness in mitigating bias while retaining crucial contextual information.
  arXiv  Detail & Related papers  (2021-06-24T17:52:43Z)
• Understanding and Countering Stereotypes: A Computational Approach to the Stereotype Content Model [4.916009028580767]
  We present a computational approach to interpreting stereotypes in text through the Stereotype Content Model (SCM) The SCM proposes that stereotypes can be understood along two primary dimensions: warmth and competence. It is known that countering stereotypes with anti-stereotypical examples is one of the most effective ways to reduce biased thinking.
  arXiv  Detail & Related papers  (2021-06-04T16:53:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.