ReNeg: Learning Negative Embedding with Reward Guidance

• URL: http://arxiv.org/abs/2412.19637v2
• Date: Sat, 08 Feb 2025 09:36:19 GMT
• Title: ReNeg: Learning Negative Embedding with Reward Guidance
• Authors: Xiaomin Li, Yixuan Liu, Takashi Isobe, Xu Jia, Qinpeng Cui, Dong Zhou, Dong Li, You He, Huchuan Lu, Zhongdao Wang, Emad Barsoum,
• Abstract summary: In text-to-image (T2I) generation applications, negative embeddings have proven to be a simple yet effective approach for enhancing generation quality. We introduce ReNeg, an end-to-end method designed to learn improved Negative embeddings guided by a Reward model.
• Score: 69.81219455975477
• License:
• Abstract: In text-to-image (T2I) generation applications, negative embeddings have proven to be a simple yet effective approach for enhancing generation quality. Typically, these negative embeddings are derived from user-defined negative prompts, which, while being functional, are not necessarily optimal. In this paper, we introduce ReNeg, an end-to-end method designed to learn improved Negative embeddings guided by a Reward model. We employ a reward feedback learning framework and integrate classifier-free guidance (CFG) into the training process, which was previously utilized only during inference, thus enabling the effective learning of negative embeddings. We also propose two strategies for learning both global and per-sample negative embeddings. Extensive experiments show that the learned negative embedding significantly outperforms null-text and handcrafted counterparts, achieving substantial improvements in human preference alignment. Additionally, the negative embedding learned within the same text embedding space exhibits strong generalization capabilities. For example, using the same CLIP text encoder, the negative embedding learned on SD1.5 can be seamlessly transferred to text-to-image or even text-to-video models such as ControlNet, ZeroScope, and VideoCrafter2, resulting in consistent performance improvements across the board.

Related papers

• Conan-embedding: General Text Embedding with More and Better Negative Samples [30.571206231457932]
  We propose a conan-embedding model, which maximizes the utilization of more and higher-quality negative examples. Our approach effectively enhances the capabilities of embedding models, currently ranking first on the Chinese leaderboard of Massive text embedding benchmark.
  arXiv  Detail & Related papers  (2024-08-28T11:18:06Z)
• Contrastive Learning with Negative Sampling Correction [52.990001829393506]
  We propose a novel contrastive learning method named Positive-Unlabeled Contrastive Learning (PUCL) PUCL treats the generated negative samples as unlabeled samples and uses information from positive samples to correct bias in contrastive loss. PUCL can be applied to general contrastive learning problems and outperforms state-of-the-art methods on various image and graph classification tasks.
  arXiv  Detail & Related papers  (2024-01-13T11:18:18Z)
• Generating Enhanced Negatives for Training Language-Based Object Detectors [86.1914216335631]
  We propose to leverage the vast knowledge built into modern generative models to automatically build negatives that are more relevant to the original data. Specifically, we use large-language-models to generate negative text descriptions, and text-to-image diffusion models to also generate corresponding negative images. Our experimental analysis confirms the relevance of the generated negative data, and its use in language-based detectors improves performance on two complex benchmarks.
  arXiv  Detail & Related papers  (2023-12-29T23:04:00Z)
• Language Model Pre-training on True Negatives [109.73819321246062]
  Discriminative pre-trained language models (PLMs) learn to predict original texts from intentionally corrupted ones. Existing PLMs simply treat all corrupted texts as equal negative without any examination. We design enhanced pre-training methods to counteract false negative predictions and encourage pre-training language models on true negatives.
  arXiv  Detail & Related papers  (2022-12-01T12:24:19Z)
• Positive-Negative Equal Contrastive Loss for Semantic Segmentation [8.664491798389662]
  Previous works commonly design plug-and-play modules and structural losses to effectively extract and aggregate the global context. We propose Positive-Negative Equal contrastive loss (PNE loss), which increases the latent impact of positive embedding on the anchor and treats the positive as well as negative sample pairs equally. We conduct comprehensive experiments and achieve state-of-the-art performance on two benchmark datasets.
  arXiv  Detail & Related papers  (2022-07-04T13:51:29Z)
• Improving Contrastive Learning of Sentence Embeddings with Case-Augmented Positives and Retrieved Negatives [17.90820242798732]
  Unsupervised contrastive learning methods still lag far behind the supervised counterparts. We propose switch-case augmentation to flip the case of the first letter of randomly selected words in a sentence. For negative samples, we sample hard negatives from the whole dataset based on a pre-trained language model.
  arXiv  Detail & Related papers  (2022-06-06T09:46:12Z)
• Robust Contrastive Learning Using Negative Samples with Diminished Semantics [23.38896719740166]
  We show that by generating carefully designed negative samples, contrastive learning can learn more robust representations. We develop two methods, texture-based and patch-based augmentations, to generate negative samples. We also analyze our method and the generated texture-based samples, showing that texture features are indispensable in classifying particular ImageNet classes.
  arXiv  Detail & Related papers  (2021-10-27T05:38:00Z)
• Incremental False Negative Detection for Contrastive Learning [95.68120675114878]
  We introduce a novel incremental false negative detection for self-supervised contrastive learning. During contrastive learning, we discuss two strategies to explicitly remove the detected false negatives. Our proposed method outperforms other self-supervised contrastive learning frameworks on multiple benchmarks within a limited compute.
  arXiv  Detail & Related papers  (2021-06-07T15:29:14Z)
• Reinforced Negative Sampling over Knowledge Graph for Recommendation [106.07209348727564]
  We develop a new negative sampling model, Knowledge Graph Policy Network (kgPolicy), which works as a reinforcement learning agent to explore high-quality negatives. kgPolicy navigates from the target positive interaction, adaptively receives knowledge-aware negative signals, and ultimately yields a potential negative item to train the recommender.
  arXiv  Detail & Related papers  (2020-03-12T12:44:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.