Understanding the Impact of Negative Prompts: When and How Do They Take Effect?

• URL: http://arxiv.org/abs/2406.02965v1
• Date: Wed, 5 Jun 2024 05:42:46 GMT
• Title: Understanding the Impact of Negative Prompts: When and How Do They Take Effect?
• Authors: Yuanhao Ban, Ruochen Wang, Tianyi Zhou, Minhao Cheng, Boqing Gong, Cho-Jui Hsieh,
• Abstract summary: This paper presents the first comprehensive study to uncover how and when negative prompts take effect. Our empirical analysis identifies two primary behaviors of negative prompts. Negative prompts can facilitate object inpainting with minimal alterations to the background via a simple adaptive algorithm.
• Score: 92.53724347718173
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The concept of negative prompts, emerging from conditional generation models like Stable Diffusion, allows users to specify what to exclude from the generated images.%, demonstrating significant practical efficacy. Despite the widespread use of negative prompts, their intrinsic mechanisms remain largely unexplored. This paper presents the first comprehensive study to uncover how and when negative prompts take effect. Our extensive empirical analysis identifies two primary behaviors of negative prompts. Delayed Effect: The impact of negative prompts is observed after positive prompts render corresponding content. Deletion Through Neutralization: Negative prompts delete concepts from the generated image through a mutual cancellation effect in latent space with positive prompts. These insights reveal significant potential real-world applications; for example, we demonstrate that negative prompts can facilitate object inpainting with minimal alterations to the background via a simple adaptive algorithm. We believe our findings will offer valuable insights for the community in capitalizing on the potential of negative prompts.

Related papers

• Optimizing Negative Prompts for Enhanced Aesthetics and Fidelity in Text-To-Image Generation [1.4138057640459576]
  We propose NegOpt, a novel method for optimizing negative prompt generation toward enhanced image generation. Our combined approach results in a substantial increase of 25% in Inception Score compared to other approaches.
  arXiv  Detail & Related papers  (2024-03-12T12:44:34Z)
• Contrastive Learning with Negative Sampling Correction [52.990001829393506]
  We propose a novel contrastive learning method named Positive-Unlabeled Contrastive Learning (PUCL) PUCL treats the generated negative samples as unlabeled samples and uses information from positive samples to correct bias in contrastive loss. PUCL can be applied to general contrastive learning problems and outperforms state-of-the-art methods on various image and graph classification tasks.
  arXiv  Detail & Related papers  (2024-01-13T11:18:18Z)
• Your Negative May not Be True Negative: Boosting Image-Text Matching with False Negative Elimination [62.18768931714238]
  We propose a novel False Negative Elimination (FNE) strategy to select negatives via sampling. The results demonstrate the superiority of our proposed false negative elimination strategy.
  arXiv  Detail & Related papers  (2023-08-08T16:31:43Z)
• SimANS: Simple Ambiguous Negatives Sampling for Dense Text Retrieval [126.22182758461244]
  We show that according to the measured relevance scores, the negatives ranked around the positives are generally more informative and less likely to be false negatives. We propose a simple ambiguous negatives sampling method, SimANS, which incorporates a new sampling probability distribution to sample more ambiguous negatives.
  arXiv  Detail & Related papers  (2022-10-21T07:18:05Z)
• Understanding CNNs from excitations [12.25690353533472]
  Saliency maps have proven to be a highly efficacious approach for explicating the decisions of Convolutional Neural Networks. We present a novel concept, termed positive and negative excitation, which enables the direct extraction of positive and negative excitation for each layer.
  arXiv  Detail & Related papers  (2022-05-02T14:27:35Z)
• Investigating the Role of Negatives in Contrastive Representation Learning [59.30700308648194]
  Noise contrastive learning is a popular technique for unsupervised representation learning. We focus on disambiguating the role of one of these parameters: the number of negative examples. We find that the results broadly agree with our theory, while our vision experiments are murkier with performance sometimes even being insensitive to the number of negatives.
  arXiv  Detail & Related papers  (2021-06-18T06:44:16Z)
• Removing Gamification: A Research Agenda [13.32560004325655]
  I offer a rapid review on the state of the art and what is known about the impact of removing gamification. Findings suggest a mix of positive and negative effects related to removing gamification. I end with a call for empirical and theoretical work on illuminating the effects that may linger after systems are un-gamified.
  arXiv  Detail & Related papers  (2021-03-10T03:59:46Z)
• AdCo: Adversarial Contrast for Efficient Learning of Unsupervised Representations from Self-Trained Negative Adversaries [55.059844800514774]
  We propose an Adrial Contrastive (AdCo) model to train representations that are hard to discriminate against positive queries. Experiment results demonstrate the proposed Adrial Contrastive (AdCo) model achieves superior performances.
  arXiv  Detail & Related papers  (2020-11-17T05:45:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.