Optimizing Negative Prompts for Enhanced Aesthetics and Fidelity in Text-To-Image Generation

• URL: http://arxiv.org/abs/2403.07605v3
• Date: Tue, 05 Nov 2024 01:11:08 GMT
• Title: Optimizing Negative Prompts for Enhanced Aesthetics and Fidelity in Text-To-Image Generation
• Authors: Michael Ogezi, Ning Shi,
• Abstract summary: We propose NegOpt, a novel method for optimizing negative prompt generation toward enhanced image generation. Our combined approach results in a substantial increase of 25% in Inception Score compared to other approaches.
• Score: 1.4138057640459576
• License:
• Abstract: In text-to-image generation, using negative prompts, which describe undesirable image characteristics, can significantly boost image quality. However, producing good negative prompts is manual and tedious. To address this, we propose NegOpt, a novel method for optimizing negative prompt generation toward enhanced image generation, using supervised fine-tuning and reinforcement learning. Our combined approach results in a substantial increase of 25% in Inception Score compared to other approaches and surpasses ground-truth negative prompts from the test set. Furthermore, with NegOpt we can preferentially optimize the metrics most important to us. Finally, we construct Negative Prompts DB (https://huggingface.co/datasets/mikeogezi/negopt_full), a publicly available dataset of negative prompts.

Related papers

• ReNeg: Learning Negative Embedding with Reward Guidance [69.81219455975477]
  In text-to-image (T2I) generation applications, negative embeddings have proven to be a simple yet effective approach for enhancing generation quality. We introduce ReNeg, an end-to-end method designed to learn improved Negative embeddings guided by a Reward model.
  arXiv  Detail & Related papers  (2024-12-27T13:31:55Z)
• Towards Unified Modeling for Positive and Negative Preferences in Sign-Aware Recommendation [13.300975621769396]
  We propose a novel textbfLight textbfSigned textbfGraph Convolution Network specifically for textbfRecommendation (textbfLSGRec) For the negative preferences within high-order heterogeneous interactions, first-order negative preferences are captured by the negative links. recommendation results are generated based on positive preferences and optimized with negative ones.
  arXiv  Detail & Related papers  (2024-03-13T05:00:42Z)
• Generating Enhanced Negatives for Training Language-Based Object Detectors [86.1914216335631]
  We propose to leverage the vast knowledge built into modern generative models to automatically build negatives that are more relevant to the original data. Specifically, we use large-language-models to generate negative text descriptions, and text-to-image diffusion models to also generate corresponding negative images. Our experimental analysis confirms the relevance of the generated negative data, and its use in language-based detectors improves performance on two complex benchmarks.
  arXiv  Detail & Related papers  (2023-12-29T23:04:00Z)
• Your Negative May not Be True Negative: Boosting Image-Text Matching with False Negative Elimination [62.18768931714238]
  We propose a novel False Negative Elimination (FNE) strategy to select negatives via sampling. The results demonstrate the superiority of our proposed false negative elimination strategy.
  arXiv  Detail & Related papers  (2023-08-08T16:31:43Z)
• DreamArtist++: Controllable One-Shot Text-to-Image Generation via Positive-Negative Adapter [63.622879199281705]
  Some example-based image generation approaches have been proposed, emphi.e. generating new concepts based on absorbing the salient features of a few input references. We propose a simple yet effective framework, namely DreamArtist, which adopts a novel positive-negative prompt-tuning learning strategy on the pre-trained diffusion model. We have conducted extensive experiments and evaluated the proposed method from image similarity (fidelity) and diversity, generation controllability, and style cloning.
  arXiv  Detail & Related papers  (2022-11-21T10:37:56Z)
• Negative Sample is Negative in Its Own Way: Tailoring Negative Sentences for Image-Text Retrieval [19.161248757493386]
  We propose our TAiloring neGative Sentences with Discrimination and Correction (TAGS-DC) to generate synthetic sentences automatically as negative samples. To keep the difficulty during training, we mutually improve the retrieval and generation through parameter sharing. In experiments, we verify the effectiveness of our model on MS-COCO and Flickr30K compared with current state-of-the-art models.
  arXiv  Detail & Related papers  (2021-11-05T09:36:41Z)
• Instance-wise Hard Negative Example Generation for Contrastive Learning in Unpaired Image-to-Image Translation [102.99799162482283]
  We present instance-wise hard Negative Example Generation for Contrastive learning in Unpaired image-to-image Translation (NEGCUT) Specifically, we train a generator to produce negative examples online. The generator is novel from two perspectives: 1) it is instance-wise which means that the generated examples are based on the input image, and 2) it can generate hard negative examples since it is trained with an adversarial loss.
  arXiv  Detail & Related papers  (2021-08-10T09:44:59Z)
• Adaptive Offline Quintuplet Loss for Image-Text Matching [102.50814151323965]
  Existing image-text matching approaches typically leverage triplet loss with online hard negatives to train the model. We propose solutions by sampling negatives offline from the whole training set. We evaluate the proposed training approach on three state-of-the-art image-text models on the MS-COCO and Flickr30K datasets.
  arXiv  Detail & Related papers  (2020-03-07T22:09:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.