EquiPrompt: Debiasing Diffusion Models via Iterative Bootstrapping in Chain of Thoughts

• URL: http://arxiv.org/abs/2406.09070v1
• Date: Thu, 13 Jun 2024 12:55:10 GMT
• Title: EquiPrompt: Debiasing Diffusion Models via Iterative Bootstrapping in Chain of Thoughts
• Authors: Zahraa Al Sahili, Ioannis Patras, Matthew Purver,
• Abstract summary: EquiPrompt is a novel method employing Chain of Thought (CoT) reasoning to reduce biases in text-to-image generative models. It integrates iterative bootstrapping and bias-aware selection to balance creativity and ethical responsibility.
• Score: 14.632649933582648
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In the domain of text-to-image generative models, the inadvertent propagation of biases inherent in training datasets poses significant ethical challenges, particularly in the generation of socially sensitive content. This paper introduces EquiPrompt, a novel method employing Chain of Thought (CoT) reasoning to reduce biases in text-to-image generative models. EquiPrompt uses iterative bootstrapping and bias-aware exemplar selection to balance creativity and ethical responsibility. It integrates iterative reasoning refinement with controlled evaluation techniques, addressing zero-shot CoT issues in sensitive contexts. Experiments on several generation tasks show EquiPrompt effectively lowers bias while maintaining generative quality, advancing ethical AI and socially responsible creative processes.Code will be publically available.

Related papers

• AITTI: Learning Adaptive Inclusive Token for Text-to-Image Generation [53.65701943405546]
  We learn adaptive inclusive tokens to shift the attribute distribution of the final generative outputs. Our method requires neither explicit attribute specification nor prior knowledge of the bias distribution. Our method achieves comparable performance to models that require specific attributes or editing directions for generation.
  arXiv  Detail & Related papers  (2024-06-18T17:22:23Z)
• Latent Directions: A Simple Pathway to Bias Mitigation in Generative AI [45.54709270833219]
  Mitigating biases in generative AI and, particularly in text-to-image models, is of high importance given their growing implications in society. Our work introduces a novel approach to achieve diverse and inclusive synthetic images by learning a direction in the latent space. It is possible to linearly combine these learned latent directions to introduce new mitigations, and if desired, integrate it with text embedding adjustments.
  arXiv  Detail & Related papers  (2024-06-10T15:13:51Z)
• Can Prompt Modifiers Control Bias? A Comparative Analysis of Text-to-Image Generative Models [8.419004940268023]
  This study examines the presence and manipulation of societal biases in leading text-to-image models: Stable Diffusion, DALL-E 3, and Adobe Firefly. Our findings reveal the challenges and potential of prompt engineering in controlling biases, highlighting the critical need for ethical AI development. This work advances AI ethics by not only revealing the nuanced dynamics of bias in text-to-image generation models but also by offering a novel framework for future research in controlling bias.
  arXiv  Detail & Related papers  (2024-06-09T00:54:57Z)
• Large Language Models as Analogical Reasoners [155.9617224350088]
  Chain-of-thought (CoT) prompting for language models demonstrates impressive performance across reasoning tasks. We introduce a new prompting approach, analogical prompting, designed to automatically guide the reasoning process of large language models.
  arXiv  Detail & Related papers  (2023-10-03T00:57:26Z)
• CoT-BERT: Enhancing Unsupervised Sentence Representation through Chain-of-Thought [3.0566617373924325]
  This paper presents CoT-BERT, an innovative method that harnesses the progressive thinking of Chain-of-supervised reasoning. We develop an advanced contrastive learning loss function and propose a novel template denoising strategy.
  arXiv  Detail & Related papers  (2023-09-20T08:42:06Z)
• Visual Chain of Thought: Bridging Logical Gaps with Multimodal Infillings [61.04460792203266]
  We introduce VCoT, a novel method that leverages chain-of-thought prompting with vision-language grounding to bridge the logical gaps within sequential data. Our method uses visual guidance to generate synthetic multimodal infillings that add consistent and novel information to reduce the logical gaps for downstream tasks.
  arXiv  Detail & Related papers  (2023-05-03T17:58:29Z)
• Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models [103.61066310897928]
  Recent text-to-image generative models have demonstrated an unparalleled ability to generate diverse and creative imagery guided by a target text prompt. While revolutionary, current state-of-the-art diffusion models may still fail in generating images that fully convey the semantics in the given text prompt. We analyze the publicly available Stable Diffusion model and assess the existence of catastrophic neglect, where the model fails to generate one or more of the subjects from the input prompt. We introduce the concept of Generative Semantic Nursing (GSN), where we seek to intervene in the generative process on the fly during inference time to improve the faithfulness
  arXiv  Detail & Related papers  (2023-01-31T18:10:38Z)
• Interpretable Data-Based Explanations for Fairness Debugging [7.266116143672294]
  Gopher is a system that produces compact, interpretable, and causal explanations for bias or unexpected model behavior. We introduce the concept of causal responsibility that quantifies the extent to which intervening on training data by removing or updating subsets of it can resolve the bias. Building on this concept, we develop an efficient approach for generating the top-k patterns that explain model bias.
  arXiv  Detail & Related papers  (2021-12-17T20:10:00Z)
• NoiER: An Approach for Training more Reliable Fine-TunedDownstream Task Models [54.184609286094044]
  We propose noise entropy regularisation (NoiER) as an efficient learning paradigm that solves the problem without auxiliary models and additional data. The proposed approach improved traditional OOD detection evaluation metrics by 55% on average compared to the original fine-tuned models.
  arXiv  Detail & Related papers  (2021-08-29T06:58:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.