RePrompt: Automatic Prompt Editing to Refine AI-Generative Art Towards Precise Expressions

• URL: http://arxiv.org/abs/2302.09466v3
• Date: Mon, 20 Mar 2023 02:34:00 GMT
• Title: RePrompt: Automatic Prompt Editing to Refine AI-Generative Art Towards Precise Expressions
• Authors: Yunlong Wang, Shuyuan Shen, Brian Y. Lim
• Abstract summary: We develop RePrompt, an automatic method to refine text prompts toward precise expression of generated images. Inspired by crowdsourced editing strategies, we curated intuitive text features, such as the number and concreteness of nouns. With model explanations of the proxy model, we curated a rubric to adjust text prompts to optimize image generation for precise emotion expression.
• Score: 9.51095076299351
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Generative AI models have shown impressive ability to produce images with text prompts, which could benefit creativity in visual art creation and self-expression. However, it is unclear how precisely the generated images express contexts and emotions from the input texts. We explored the emotional expressiveness of AI-generated images and developed RePrompt, an automatic method to refine text prompts toward precise expression of the generated images. Inspired by crowdsourced editing strategies, we curated intuitive text features, such as the number and concreteness of nouns, and trained a proxy model to analyze the feature effects on the AI-generated image. With model explanations of the proxy model, we curated a rubric to adjust text prompts to optimize image generation for precise emotion expression. We conducted simulation and user studies, which showed that RePrompt significantly improves the emotional expressiveness of AI-generated images, especially for negative emotions.

Related papers

• Affective Image Editing: Shaping Emotional Factors via Text Descriptions [46.13506671212571]
  We introduce AIEdiT for Affective Image Editing using Text descriptions.<n>We build the continuous emotional spectrum and extract nuanced emotional requests.<n>AIEdiT achieves superior performance, effectively reflecting users' emotional requests.
  arXiv  Detail & Related papers  (2025-05-24T13:46:57Z)
• RePrompt: Reasoning-Augmented Reprompting for Text-to-Image Generation via Reinforcement Learning [88.14234949860105]
  RePrompt is a novel reprompting framework that introduces explicit reasoning into the prompt enhancement process via reinforcement learning.<n>Our approach enables end-to-end training without human-annotated data.
  arXiv  Detail & Related papers  (2025-05-23T06:44:26Z)
• EmotiCrafter: Text-to-Emotional-Image Generation based on Valence-Arousal Model [23.26111054485357]
  We introduce the new task of continuous emotional image content generation (C-EICG) We present EmotiCrafter, an emotional image generation model that generates images based on text prompts and Valence-Arousal values.
  arXiv  Detail & Related papers  (2025-01-10T04:41:37Z)
• Dynamic Prompt Optimizing for Text-to-Image Generation [63.775458908172176]
  We introduce the textbfPrompt textbfAuto-textbfEditing (PAE) method to improve text-to-image generative models. We employ an online reinforcement learning strategy to explore the weights and injection time steps of each word, leading to the dynamic fine-control prompts.
  arXiv  Detail & Related papers  (2024-04-05T13:44:39Z)
• EmoGen: Emotional Image Content Generation with Text-to-Image Diffusion Models [11.901294654242376]
  We introduce Emotional Image Content Generation (EICG), a new task to generate semantic-clear and emotion-faithful images given emotion categories. Specifically, we propose an emotion space and construct a mapping network to align it with the powerful Contrastive Language-Image Pre-training (CLIP) space. Our method outperforms the state-of-the-art text-to-image approaches both quantitatively and qualitatively.
  arXiv  Detail & Related papers  (2024-01-09T15:23:21Z)
• Prompt Expansion for Adaptive Text-to-Image Generation [51.67811570987088]
  This paper proposes a Prompt Expansion framework that helps users generate high-quality, diverse images with less effort. The Prompt Expansion model takes a text query as input and outputs a set of expanded text prompts. We conduct a human evaluation study that shows that images generated through Prompt Expansion are more aesthetically pleasing and diverse than those generated by baseline methods.
  arXiv  Detail & Related papers  (2023-12-27T21:12:21Z)
• A Portrait of Emotion: Empowering Self-Expression through AI-Generated Art [0.0]
  We investigated the potential and limitations of generative artificial intelligence (AI) in reflecting the authors' cognitive processes through creative expression. Results show a preference for images based on the descriptions of the authors' emotions over the main events. Our research framework with generative AIs can help design AI-based interventions in related fields.
  arXiv  Detail & Related papers  (2023-04-26T06:54:53Z)
• Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models [103.61066310897928]
  Recent text-to-image generative models have demonstrated an unparalleled ability to generate diverse and creative imagery guided by a target text prompt. While revolutionary, current state-of-the-art diffusion models may still fail in generating images that fully convey the semantics in the given text prompt. We analyze the publicly available Stable Diffusion model and assess the existence of catastrophic neglect, where the model fails to generate one or more of the subjects from the input prompt. We introduce the concept of Generative Semantic Nursing (GSN), where we seek to intervene in the generative process on the fly during inference time to improve the faithfulness
  arXiv  Detail & Related papers  (2023-01-31T18:10:38Z)
• Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translation [10.39028769374367]
  We present a new framework that takes text-to-image synthesis to the realm of image-to-image translation. Our method harnesses the power of a pre-trained text-to-image diffusion model to generate a new image that complies with the target text.
  arXiv  Detail & Related papers  (2022-11-22T20:39:18Z)
• Visualize Before You Write: Imagination-Guided Open-Ended Text Generation [68.96699389728964]
  We propose iNLG that uses machine-generated images to guide language models in open-ended text generation. Experiments and analyses demonstrate the effectiveness of iNLG on open-ended text generation tasks.
  arXiv  Detail & Related papers  (2022-10-07T18:01:09Z)
• ViNTER: Image Narrative Generation with Emotion-Arc-Aware Transformer [59.05857591535986]
  We propose a model called ViNTER to generate image narratives that focus on time series representing varying emotions as "emotion arcs" We present experimental results of both manual and automatic evaluations.
  arXiv  Detail & Related papers  (2022-02-15T10:53:08Z)
• Altering Facial Expression Based on Textual Emotion [0.19573380763700707]
  We aim to change the facial expression in an image using the Generative Adversarial Network (GAN) We extend our work by remodeling facial expressions in an image indicated by the emotion from a given text. As a demonstration of our working pipeline, we create an application prototype of a blog that regenerates the profile picture with different expressions.
  arXiv  Detail & Related papers  (2021-12-02T17:52:25Z)
• Facial Expression Editing with Continuous Emotion Labels [76.36392210528105]
  Deep generative models have achieved impressive results in the field of automated facial expression editing. We propose a model that can be used to manipulate facial expressions in facial images according to continuous two-dimensional emotion labels.
  arXiv  Detail & Related papers  (2020-06-22T13:03:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.