Optimizing Negative Prompts for Enhanced Aesthetics and Fidelity in Text-To-Image Generation

• URL: http://arxiv.org/abs/2403.07605v3
• Date: Tue, 05 Nov 2024 01:11:08 GMT
• Title: Optimizing Negative Prompts for Enhanced Aesthetics and Fidelity in Text-To-Image Generation
• Authors: Michael Ogezi, Ning Shi,
• Abstract summary: We propose NegOpt, a novel method for optimizing negative prompt generation toward enhanced image generation. Our combined approach results in a substantial increase of 25% in Inception Score compared to other approaches.
• Score: 1.4138057640459576
• License:
• Abstract: In text-to-image generation, using negative prompts, which describe undesirable image characteristics, can significantly boost image quality. However, producing good negative prompts is manual and tedious. To address this, we propose NegOpt, a novel method for optimizing negative prompt generation toward enhanced image generation, using supervised fine-tuning and reinforcement learning. Our combined approach results in a substantial increase of 25% in Inception Score compared to other approaches and surpasses ground-truth negative prompts from the test set. Furthermore, with NegOpt we can preferentially optimize the metrics most important to us. Finally, we construct Negative Prompts DB (https://huggingface.co/datasets/mikeogezi/negopt_full), a publicly available dataset of negative prompts.

Related papers

• Understanding the Impact of Negative Prompts: When and How Do They Take Effect? [92.53724347718173]
  This paper presents the first comprehensive study to uncover how and when negative prompts take effect. Our empirical analysis identifies two primary behaviors of negative prompts. Negative prompts can facilitate object inpainting with minimal alterations to the background via a simple adaptive algorithm.
  arXiv  Detail & Related papers  (2024-06-05T05:42:46Z)
• Towards Unified Modeling for Positive and Negative Preferences in Sign-Aware Recommendation [13.300975621769396]
  We propose a novel textbfLight textbfSigned textbfGraph Convolution Network specifically for textbfRecommendation (textbfLSGRec) For the negative preferences within high-order heterogeneous interactions, first-order negative preferences are captured by the negative links. recommendation results are generated based on positive preferences and optimized with negative ones.
  arXiv  Detail & Related papers  (2024-03-13T05:00:42Z)
• Generating Enhanced Negatives for Training Language-Based Object Detectors [86.1914216335631]
  We propose to leverage the vast knowledge built into modern generative models to automatically build negatives that are more relevant to the original data. Specifically, we use large-language-models to generate negative text descriptions, and text-to-image diffusion models to also generate corresponding negative images. Our experimental analysis confirms the relevance of the generated negative data, and its use in language-based detectors improves performance on two complex benchmarks.
  arXiv  Detail & Related papers  (2023-12-29T23:04:00Z)
• Your Negative May not Be True Negative: Boosting Image-Text Matching with False Negative Elimination [62.18768931714238]
  We propose a novel False Negative Elimination (FNE) strategy to select negatives via sampling. The results demonstrate the superiority of our proposed false negative elimination strategy.
  arXiv  Detail & Related papers  (2023-08-08T16:31:43Z)
• DreamArtist: Towards Controllable One-Shot Text-to-Image Generation via Positive-Negative Prompt-Tuning [85.10894272034135]
  Large-scale text-to-image generation models have achieved remarkable progress in synthesizing high-quality, feature-rich images with high resolution guided by texts. Recent attempts have employed fine-tuning or prompt-tuning strategies to teach the pre-trained diffusion model novel concepts from a reference image set. We present a simple yet effective method called DreamArtist, which employs a positive-negative prompt-tuning learning strategy.
  arXiv  Detail & Related papers  (2022-11-21T10:37:56Z)
• Exploring Negatives in Contrastive Learning for Unpaired Image-to-Image Translation [12.754320302262533]
  We introduce a new negative Pruning technology for Unpaired image-to-image Translation (PUT) by sparsifying and ranking the patches. The proposed algorithm is efficient, flexible and enables the model to learn essential information between corresponding patches stably.
  arXiv  Detail & Related papers  (2022-04-23T08:31:18Z)
• Negative Sample is Negative in Its Own Way: Tailoring Negative Sentences for Image-Text Retrieval [19.161248757493386]
  We propose our TAiloring neGative Sentences with Discrimination and Correction (TAGS-DC) to generate synthetic sentences automatically as negative samples. To keep the difficulty during training, we mutually improve the retrieval and generation through parameter sharing. In experiments, we verify the effectiveness of our model on MS-COCO and Flickr30K compared with current state-of-the-art models.
  arXiv  Detail & Related papers  (2021-11-05T09:36:41Z)
• Instance-wise Hard Negative Example Generation for Contrastive Learning in Unpaired Image-to-Image Translation [102.99799162482283]
  We present instance-wise hard Negative Example Generation for Contrastive learning in Unpaired image-to-image Translation (NEGCUT) Specifically, we train a generator to produce negative examples online. The generator is novel from two perspectives: 1) it is instance-wise which means that the generated examples are based on the input image, and 2) it can generate hard negative examples since it is trained with an adversarial loss.
  arXiv  Detail & Related papers  (2021-08-10T09:44:59Z)
• Adaptive Offline Quintuplet Loss for Image-Text Matching [102.50814151323965]
  Existing image-text matching approaches typically leverage triplet loss with online hard negatives to train the model. We propose solutions by sampling negatives offline from the whole training set. We evaluate the proposed training approach on three state-of-the-art image-text models on the MS-COCO and Flickr30K datasets.
  arXiv  Detail & Related papers  (2020-03-07T22:09:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.