PromptEvolver: Prompt Inversion through Evolutionary Optimization in Natural-Language Space

• URL: http://arxiv.org/abs/2604.06061v1
• Date: Fri, 03 Apr 2026 17:00:49 GMT
• Title: PromptEvolver: Prompt Inversion through Evolutionary Optimization in Natural-Language Space
• Abstract summary: We present PromptEvolver, a prompt inversion approach that generates natural-language prompts while achieving high-fidelity reconstructions of the target image.<n>Our method uses a genetic algorithm to optimize the prompt, leveraging a strong vision-language model to guide the evolution process.
• Score: 20.416910591388618
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Text-to-image generation has progressed rapidly, but faithfully generating complex scenes requires extensive trial-and-error to find the exact prompt. In the prompt inversion task, the goal is to recover a textual prompt that can faithfully reconstruct a given target image. Currently, existing methods frequently yield suboptimal reconstructions and produce unnatural, hard-to-interpret prompts that hinder transparency and controllability. In this work, we present PromptEvolver, a prompt inversion approach that generates natural-language prompts while achieving high-fidelity reconstructions of the target image. Our method uses a genetic algorithm to optimize the prompt, leveraging a strong vision-language model to guide the evolution process. Importantly, it works on black-box generation models by requiring only image outputs. Finally, we evaluate PromptEvolver across multiple prompt inversion benchmarks and show that it consistently outperforms competing methods.

Related papers

• PROVE: Training-Free Prompt Recovery using Verifiable Evidence [5.5813258856679555]
  We introduce PROVE (Prompt Recovery with Verified Evidence), a training-free, black-box prompt inversion attack.<n>It reconstructs prompts by composing verifiable scene descriptions rather than optimizing token sequences.<n>The resulting prompts are fully auditable, with every recovered claim grounded in explicit image evidence.
  arXiv  Detail & Related papers  (2026-08-13T18:08:00Z)
• GhostPrompt: Cross-Image Adversarial Prompt for Vision-Language Models [19.133064005999838]
  GhostPrompt is an adversarial prompt that is optimized once and reused to steer VLM outputs toward attacker-specified responses across diverse images.<n>Our method achieves an improvement of over 30% in attack success rates compared to state-of-the-art (SoTA) baselines, while reducing computation time by 70%.
  arXiv  Detail & Related papers  (2026-07-22T02:33:56Z)
• EDITOR: Effective and Interpretable Prompt Inversion for Text-to-Image Diffusion Models [41.63451923844824]
  We propose a prompt inversion technique called sys for text-to-image diffusion models.<n>Our method outperforms existing methods in terms of image similarity, textual alignment, prompt interpretability and generalizability.
  arXiv  Detail & Related papers  (2025-06-03T16:44:15Z)
• Image Generation from Contextually-Contradictory Prompts [50.999420029656214]
  We propose a stage-aware prompt decomposition framework that guides the denoising process using a sequence of proxy prompts.<n>Our method enables fine-grained semantic control and accurate image generation in the presence of contextual contradictions.
  arXiv  Detail & Related papers  (2025-06-02T17:48:12Z)
• RePrompt: Reasoning-Augmented Reprompting for Text-to-Image Generation via Reinforcement Learning [88.14234949860105]
  RePrompt is a novel reprompting framework that introduces explicit reasoning into the prompt enhancement process via reinforcement learning.<n>Our approach enables end-to-end training without human-annotated data.
  arXiv  Detail & Related papers  (2025-05-23T06:44:26Z)
• Reverse Prompt: Cracking the Recipe Inside Text-to-Image Generation [87.34490419583558]
  We propose a method known as automatic reverse prompt optimization (ARPO)<n>Our method refines an initial prompt into a high-quality prompt through an iteratively imitative gradient prompt optimization process.<n>We can easily create novel images with diverse styles and content by directly editing these reverse prompts.
  arXiv  Detail & Related papers  (2025-03-25T02:08:05Z)
• Dynamic Prompt Optimizing for Text-to-Image Generation [63.775458908172176]
  We introduce the textbfPrompt textbfAuto-textbfEditing (PAE) method to improve text-to-image generative models. We employ an online reinforcement learning strategy to explore the weights and injection time steps of each word, leading to the dynamic fine-control prompts.
  arXiv  Detail & Related papers  (2024-04-05T13:44:39Z)
• Prompt Expansion for Adaptive Text-to-Image Generation [51.67811570987088]
  This paper proposes a Prompt Expansion framework that helps users generate high-quality, diverse images with less effort. The Prompt Expansion model takes a text query as input and outputs a set of expanded text prompts. We conduct a human evaluation study that shows that images generated through Prompt Expansion are more aesthetically pleasing and diverse than those generated by baseline methods.
  arXiv  Detail & Related papers  (2023-12-27T21:12:21Z)
• Prompting Hard or Hardly Prompting: Prompt Inversion for Text-to-Image Diffusion Models [46.18013380882767]
  This work focuses on inverting the diffusion model to obtain interpretable language prompts directly. We leverage the findings that different timesteps of the diffusion process cater to different levels of detail in an image. We show that our approach can identify semantically interpretable and meaningful prompts for a target image.
  arXiv  Detail & Related papers  (2023-12-19T18:47:30Z)
• SUR-adapter: Enhancing Text-to-Image Pre-trained Diffusion Models with Large Language Models [56.88192537044364]
  We propose a simple-yet-effective parameter-efficient fine-tuning approach called the Semantic Understanding and Reasoning adapter (SUR-adapter) for pre-trained diffusion models. Our approach can make text-to-image diffusion models easier to use with better user experience.
  arXiv  Detail & Related papers  (2023-05-09T05:48:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.