The Bias Amplification Paradox in Text-to-Image Generation

• URL: http://arxiv.org/abs/2308.00755v2
• Date: Wed, 15 Nov 2023 08:51:11 GMT
• Title: The Bias Amplification Paradox in Text-to-Image Generation
• Abstract summary: We study bias amplification in the text-to-image domain using Stable Diffusion. We find that the model appears to amplify gender-occupation biases found in the training data considerably. amplification can be largely attributed to discrepancies between training captions and model prompts.
• Score: 23.7496998646073
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Bias amplification is a phenomenon in which models exacerbate biases or stereotypes present in the training data. In this paper, we study bias amplification in the text-to-image domain using Stable Diffusion by comparing gender ratios in training vs. generated images. We find that the model appears to amplify gender-occupation biases found in the training data (LAION) considerably. However, we discover that amplification can be largely attributed to discrepancies between training captions and model prompts. For example, an inherent difference is that captions from the training data often contain explicit gender information while our prompts do not, which leads to a distribution shift and consequently inflates bias measures. Once we account for distributional differences between texts used for training and generation when evaluating amplification, we observe that amplification decreases drastically. Our findings illustrate the challenges of comparing biases in models and their training data, and highlight confounding factors that impact analyses.

Related papers

• Selective Underfitting in Diffusion Models [62.51850508420274]
  We argue that diffusion models underfit the empirical score due to training-time inductive biases.<n>Instead of underfitting the score everywhere, better diffusion models more accurately approximate the score in certain regions of input space, while underfitting it in others.<n>Our results establish that selective underfitting is essential for understanding diffusion models, yielding new, testable insights into their generalization and generative performance.
  arXiv  Detail & Related papers  (2025-10-01T19:06:20Z)
• How far can bias go? -- Tracing bias from pretraining data to alignment [54.51310112013655]
  This study examines the correlation between gender-occupation bias in pre-training data and their manifestation in LLMs. Our findings reveal that biases present in pre-training data are amplified in model outputs.
  arXiv  Detail & Related papers  (2024-11-28T16:20:25Z)
• Debiasing Classifiers by Amplifying Bias with Latent Diffusion and Large Language Models [9.801159950963306]
  We introduce DiffuBias, a novel pipeline for text-to-image generation that enhances classifier robustness by generating bias-conflict samples. DrouBias is the first approach leveraging a stable diffusion model to generate bias-conflict samples in debiasing tasks. Our comprehensive experimental evaluations demonstrate that DiffuBias achieves state-of-the-art performance on benchmark datasets.
  arXiv  Detail & Related papers  (2024-11-25T04:11:16Z)
• Less can be more: representational vs. stereotypical gender bias in facial expression recognition [3.9698529891342207]
  Machine learning models can inherit biases from their training data, leading to discriminatory or inaccurate predictions. This paper investigates the propagation of demographic biases from datasets into machine learning models. We focus on the gender demographic component, analyzing two types of bias: representational and stereotypical.
  arXiv  Detail & Related papers  (2024-06-25T09:26:49Z)
• Data Attribution for Diffusion Models: Timestep-induced Bias in Influence Estimation [53.27596811146316]
  Diffusion models operate over a sequence of timesteps instead of instantaneous input-output relationships in previous contexts. We present Diffusion-TracIn that incorporates this temporal dynamics and observe that samples' loss gradient norms are highly dependent on timestep. We introduce Diffusion-ReTrac as a re-normalized adaptation that enables the retrieval of training samples more targeted to the test sample of interest.
  arXiv  Detail & Related papers  (2024-01-17T07:58:18Z)
• Evaluating Bias and Fairness in Gender-Neutral Pretrained Vision-and-Language Models [23.65626682262062]
  We quantify bias amplification in pretraining and after fine-tuning on three families of vision-and-language models. Overall, we find that bias amplification in pretraining and after fine-tuning are independent.
  arXiv  Detail & Related papers  (2023-10-26T16:19:19Z)
• Understanding and Mitigating Copying in Diffusion Models [53.03978584040557]
  Images generated by diffusion models like Stable Diffusion are increasingly widespread. Recent works and even lawsuits have shown that these models are prone to replicating their training data, unbeknownst to the user.
  arXiv  Detail & Related papers  (2023-05-31T17:58:02Z)
• Discffusion: Discriminative Diffusion Models as Few-shot Vision and Language Learners [88.07317175639226]
  We propose a novel approach, Discriminative Stable Diffusion (DSD), which turns pre-trained text-to-image diffusion models into few-shot discriminative learners. Our approach mainly uses the cross-attention score of a Stable Diffusion model to capture the mutual influence between visual and textual information.
  arXiv  Detail & Related papers  (2023-05-18T05:41:36Z)
• Analyzing Bias in Diffusion-based Face Generation Models [75.80072686374564]
  Diffusion models are increasingly popular in synthetic data generation and image editing applications. We investigate the presence of bias in diffusion-based face generation models with respect to attributes such as gender, race, and age. We examine how dataset size affects the attribute composition and perceptual quality of both diffusion and Generative Adversarial Network (GAN) based face generation models.
  arXiv  Detail & Related papers  (2023-05-10T18:22:31Z)
• Fair Diffusion: Instructing Text-to-Image Generation Models on Fairness [15.059419033330126]
  We present a novel strategy, called Fair Diffusion, to attenuate biases after the deployment of generative text-to-image models. Specifically, we demonstrate shifting a bias, based on human instructions, in any direction yielding arbitrarily new proportions for, e.g., identity groups. This introduced control enables instructing generative image models on fairness, with no data filtering and additional training required.
  arXiv  Detail & Related papers  (2023-02-07T18:25:28Z)
• Deep Learning on a Healthy Data Diet: Finding Important Examples for Fairness [15.210232622716129]
  Data-driven predictive solutions predominant in commercial applications tend to suffer from biases and stereotypes. Data augmentation reduces gender bias by adding counterfactual examples to the training dataset. We show that some of the examples in the augmented dataset can be not important or even harmful for fairness.
  arXiv  Detail & Related papers  (2022-11-20T22:42:30Z)
• Improving Robustness by Augmenting Training Sentences with Predicate-Argument Structures [62.562760228942054]
  Existing approaches to improve robustness against dataset biases mostly focus on changing the training objective. We propose to augment the input sentences in the training data with their corresponding predicate-argument structures. We show that without targeting a specific bias, our sentence augmentation improves the robustness of transformer models against multiple biases.
  arXiv  Detail & Related papers  (2020-10-23T16:22:05Z)
• Mitigating Gender Bias Amplification in Distribution by Posterior Regularization [75.3529537096899]
  We investigate the gender bias amplification issue from the distribution perspective. We propose a bias mitigation approach based on posterior regularization. Our study sheds the light on understanding the bias amplification.
  arXiv  Detail & Related papers  (2020-05-13T11:07:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.