APE: Agentic Prompt Enhancer for Image Generation and Editing

• URL: http://arxiv.org/abs/2606.00204v1
• Date: Fri, 29 May 2026 17:59:31 GMT
• Title: APE: Agentic Prompt Enhancer for Image Generation and Editing
• Abstract summary: Agentic Prompt Enhancer (APE) is a lightweight framework that post-trains small language models (SLMs) as prompt-enhancement agents.<n>APE supports both single-agent rewriting and role-specialized multi-agent enhancement.<n>Its single-agent instantiation, SAPE, rewrites the prompt in one pass, while its multi-agent instantiation, MAPE, decomposes enhancement into a router--rewriter--composer process.
• Score: 58.43341902967105
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Natural language has become a powerful interface for image generation and editing, yet text-guided visual systems remain highly sensitive to prompt formulation. Semantically similar requests can produce different outputs depending on wording, specificity, and how explicitly visual constraints are stated, motivating prompt enhancement as a trainable component rather than a peripheral user choice. Existing strong enhancers often rely on large, proprietary LLMs such as ChatGPT or Gemini, adding cost, latency, and deployment dependence to the visual generation pipeline. We propose Agentic Prompt Enhancer (APE), a lightweight framework that post-trains small language models (SLMs) as prompt-enhancement agents. APE supports both single-agent rewriting and role-specialized multi-agent enhancement. Its single-agent instantiation, SAPE, rewrites the prompt in one pass, while its multi-agent instantiation, MAPE, decomposes enhancement into a router--rewriter--composer process for handling compositional constraints over objects, attributes, spatial relations, and edits. With task-aware rewards and post-training protocols, APE improves visual alignment and prompt following without modifying the downstream visual model. Experiments on challenging image generation and editing benchmarks demonstrate that post-trained small prompt enhancers reliably outperform their base counterparts, narrowing the gap to closed-source prompt enhancers; in addition, MAPE proves particularly strong on complex compositional tasks within these benchmarks.

Related papers

• WeAgent-MMGenEdit: A Full-Stack Recipe for Multimodal Agentic Image Generation and Editing [51.8784267764917]
  We present WeAgent-MMGenEdit, a full-stack recipe including a multimodal harness, a scalable data construction pipeline, a comprehensive benchmark, and post-training methods for the agent policy and image backend.<n>We first introduce WeAgent-Harness, a multimodal runtime with persistent evidence management and dedicated verification and integration tools that organize retrieved multimodal evidence into a dense carrier. Upon this, we develop a scalable pipeline for prompt synthesis and agentic trajectory collection, yielding 23K supervised trajectories and 14.7K RL tasks with three-layer verifiable checklists.
  arXiv  Detail & Related papers  (2026-09-04T14:12:03Z)
• MSRAMIE: Multimodal Structured Reasoning Agent for Multi-instruction Image Editing [6.097400596282323]
  We propose MSRAMIE, a training-free agent framework built on Multimodal Large Language Model (MLLM)<n>MSRAMIE takes existing editing models as plug-in components and handle multi-instruction tasks via structured multimodal reasoning.<n> Experiments show that as the instruction complexity increases, MSRAMIE can improve instruction following over 15% and increases the probability of finishing all modifications in a single run over 100%.
  arXiv  Detail & Related papers  (2026-03-17T09:05:16Z)
• coDrawAgents: A Multi-Agent Dialogue Framework for Compositional Image Generation [48.027946344020314]
  We propose coDrawAgents, an interactive multi-agent dialogue framework with four specialized agents.<n>The Interpreter decides between a direct text-to-image pathway and a layout-aware multi-agent process.<n>The Planner proposes layouts for objects with the same semantic priority level while grounding decisions in the evolving visual context.<n>The Checker introduces an explicit error-correction mechanism by validating spatial consistency and attribute alignment.<n>The Painter synthesizes the image step by step, incorporating newly planned objects into the canvas to provide richer context for subsequent iterations.
  arXiv  Detail & Related papers  (2026-03-13T09:32:06Z)
• How Well Do Models Follow Visual Instructions? VIBE: A Systematic Benchmark for Visual Instruction-Driven Image Editing [56.60465182650588]
  We introduce three-level interaction hierarchy that captures deictic grounding, morphological manipulation, and causal reasoning.<n>We propose a robust LMM-as-a-judge evaluation framework with task-specific metrics to enable scalable and fine-grained assessment.<n>We find that proprietary models exhibit early-stage visual instruction-following capabilities and consistently outperform open-source models.
  arXiv  Detail & Related papers  (2026-02-02T09:24:45Z)
• PromptSculptor: Multi-Agent Based Text-to-Image Prompt Optimization [4.133498001057646]
  To generate high-quality images, users must craft detailed prompts specifying scene, style, and context.<n>We propose PromptSculptor, a novel multi-agent framework that automates this iterative prompt optimization process.<n>Our system decomposes the task into four specialized agents that work collaboratively to transform a short, vague user prompt into a comprehensive, refined prompt.
  arXiv  Detail & Related papers  (2025-09-15T20:52:11Z)
• VSC: Visual Search Compositional Text-to-Image Diffusion Model [15.682990658945682]
  We introduce a novel compositional generation method that leverages pairwise image embeddings to improve attribute-object binding.<n>Our approach decomposes complex prompts into sub-prompts, generates corresponding images, and computes visual prototypes that fuse with text embeddings to enhance representation.<n>Our approaches outperform existing compositional text-to-image diffusion models on the benchmark T2I CompBench, achieving better image quality, evaluated by humans, and emerging robustness under scaling number of binding pairs in the prompt.
  arXiv  Detail & Related papers  (2025-05-02T08:31:43Z)
• Iris: Breaking GUI Complexity with Adaptive Focus and Self-Refining [67.87810796668981]
  Information-Sensitive Cropping (ISC) and Self-Refining Dual Learning (SRDL)<n>Iris achieves state-of-the-art performance across multiple benchmarks with only 850K GUI annotations.<n>These improvements translate to significant gains in both web and OS agent downstream tasks.
  arXiv  Detail & Related papers  (2024-12-13T18:40:10Z)
• ForgeryGPT: Multimodal Large Language Model For Explainable Image Forgery Detection and Localization [49.12958154544838]
  ForgeryGPT is a novel framework that advances the Image Forgery Detection and localization task.<n>It captures high-order correlations of forged images from diverse linguistic feature spaces.<n>It enables explainable generation and interactive dialogue through a newly customized Large Language Model (LLM) architecture.
  arXiv  Detail & Related papers  (2024-10-14T07:56:51Z)
• Divide and Conquer: Language Models can Plan and Self-Correct for Compositional Text-to-Image Generation [72.6168579583414]
  CompAgent is a training-free approach for compositional text-to-image generation with a large language model (LLM) agent as its core. Our approach achieves more than 10% improvement on T2I-CompBench, a comprehensive benchmark for open-world compositional T2I generation.
  arXiv  Detail & Related papers  (2024-01-28T16:18:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.