IEA: Amateur-Friendly Conversational Image Editing Agent via Three Stages of Multitask Alignment

• URL: http://arxiv.org/abs/2606.08016v1
• Date: Sat, 06 Jun 2026 07:11:08 GMT
• Title: IEA: Amateur-Friendly Conversational Image Editing Agent via Three Stages of Multitask Alignment
• Abstract summary: We propose IEA, a conversational Image Editing Agent that learns to operate parameterized tools in an explicit, interpretable action space.<n>IEA is trained via a three-stage multitask pipeline: SFT on distilled expert edits, GRPO with rewards for likeness improvement, tool usefulness, and intent summarization, and (3) large-scale synthetic fine-tuning to jointly master image editing, refinement, and user intent summarization.
• Score: 16.199788035961564
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Current image editing software often hinges on fixed filters or expert tuning, leaving a gap between amateur users' intent and outcomes. Creations by generative models may contain artifacts, implausible details, or stylistic drift away from photorealism and offer little insight into why an edit was made. We propose IEA, a conversational Image Editing Agent that learns to operate parameterized tools in an explicit, interpretable action space. IEA is trained via a three-stage multitask pipeline: (1) SFT on distilled expert edits, (2) GRPO with rewards for likeness improvement, tool usefulness, and intent summarization, and (3) large-scale synthetic fine-tuning to jointly master image editing, refinement, and user intent summarization. By manipulating 16 editing tools step by step, IEA produces transparent edit traces that can be inspected and debugged. In quantitative experiments, it attains a lower pixel distance on the edit task and a higher ROUGE-L on the summary task than strong baselines. In user studies, it ranks best among tool-calling methods for instruction following while surpassing generative methods in overall perceptual quality. Our results validate interpretable, tool-centric VLMs as a reliable path to human instruction-guided image retouching.

Related papers

• Multi-Tool Image Editing Attribution in Facial Forgery [28.69642908503663]
  Multi-Tool Image Editing Attribution (MIEA), which aims to identify multiple editing tools involved in a multi-tool edited facial image.<n>Inspired by the findings from data analysis, we design DPEC, a multi-tool attribution method that can capture distinguishable, locality-aware editing tool traces.<n>Experiments show DPEC outperforms nine methods for facial images edited in at most five steps.
  arXiv  Detail & Related papers  (2026-09-02T15:51:26Z)
• CanvasAgent: Enabling Complex Image Creation and Editing via Visual Tool Orchestration [11.992882604393003]
  We introduce CanvasCraft, a large-scale multimodal tool-use dataset for complex image creation and editing.<n>We also present CanvasAgent, a toolaugmented multimodal agent that learns to orchestrate heterogeneous visual tools through multi-turn interaction.
  arXiv  Detail & Related papers  (2026-07-06T04:57:18Z)
• Rethinking Scribble-Guided Image Editing: Generalization, Instruction Adherence, and Multi-Tasking [14.173267480504842]
  Scribble-guided image editing allows users to combine simple scribble annotations with text prompts to specify both where and how an image should be edited.<n>Existing models still exhibit unstable performance under this paradigm, especially in multi-task scenarios.<n>We conduct empirical studies using an open-source editing model and reveal an asymmetry in generalization.
  arXiv  Detail & Related papers  (2026-05-25T08:20:23Z)
• MIRA: Multimodal Iterative Reasoning Agent for Image Editing [48.41212094929379]
  We propose MIRA (Multimodal Iterative Reasoning Agent), a lightweight, plug-and-play multimodal reasoning agent.<n>Instead of issuing a single prompt or static plan, MIRA predicts atomic edit instructions step by step, using visual feedback to make its decisions.<n>Our 150K multimodal tool-use dataset, MIRA-Editing, combined with a two-stage SFT + GRPO training pipeline, enables MIRA to perform reasoning and editing over complex editing instructions.
  arXiv  Detail & Related papers  (2025-11-26T06:13:32Z)
• UniEdit-I: Training-free Image Editing for Unified VLM via Iterative Understanding, Editing and Verifying [64.5307229755533]
  We introduce a novel training-free framework named UniEdit-I to enable the unified VLM with image editing capability.<n>We implement our method based on the latest BLIP3-o and achieved state-of-the-art (SOTA) performance on the GEdit-Bench benchmark.
  arXiv  Detail & Related papers  (2025-08-05T06:42:09Z)
• Beyond Editing Pairs: Fine-Grained Instructional Image Editing via Multi-Scale Learnable Regions [20.617718631292696]
  We develop a novel paradigm for instruction-driven image editing that leverages widely available and enormous text-image pairs.<n>Our approach introduces a multi-scale learnable region to localize and guide the editing process.<n>By treating the alignment between images and their textual descriptions as supervision and learning to generate task-specific editing regions, our method achieves high-fidelity, precise, and instruction-consistent image editing.
  arXiv  Detail & Related papers  (2025-05-25T22:40:59Z)
• MonetGPT: Solving Puzzles Enhances MLLMs' Image Retouching Skills [37.48977077142813]
  We show that a multimodal large language model (MLLM) can be taught to critique raw photographs.<n>We demonstrate that MLLMs can be first made aware of the underlying image processing operations.<n>We then synthesize a reasoning dataset by procedurally manipulating expert-edited photos.
  arXiv  Detail & Related papers  (2025-05-09T16:38:27Z)
• Lost in Edits? A $λ$-Compass for AIGC Provenance [119.95562081325552]
  We propose a novel latent-space attribution method that robustly identifies and differentiates authentic outputs from manipulated ones.<n>LambdaTracer is effective across diverse iterative editing processes, whether automated through text-guided editing tools such as InstructPix2Pix or performed manually with editing software such as Adobe Photoshop.
  arXiv  Detail & Related papers  (2025-02-05T06:24:25Z)
• UIP2P: Unsupervised Instruction-based Image Editing via Edit Reversibility Constraint [87.20985852686785]
  We propose an unsupervised instruction-based image editing approach that removes the need for ground-truth edited images during training.<n>Our approach addresses these challenges by introducing a novel editing mechanism called Edit Reversibility Constraint (ERC), which applies forward and reverse edits in one training step.<n>This allows us to bypass the need for ground-truth edited images and unlock training for the first time on datasets comprising either real image-caption pairs or image-caption-instruction triplets.
  arXiv  Detail & Related papers  (2024-12-19T18:59:58Z)
• Real-time 3D-aware Portrait Editing from a Single Image [111.27169315556444]
  3DPE can edit a face image following given prompts, like reference images or text descriptions. A lightweight module is distilled from a 3D portrait generator and a text-to-image model.
  arXiv  Detail & Related papers  (2024-02-21T18:36:26Z)
• Emu Edit: Precise Image Editing via Recognition and Generation Tasks [62.95717180730946]
  We present Emu Edit, a multi-task image editing model which sets state-of-the-art results in instruction-based image editing. We train it to multi-task across an unprecedented range of tasks, such as region-based editing, free-form editing, and Computer Vision tasks. We show that Emu Edit can generalize to new tasks, such as image inpainting, super-resolution, and compositions of editing tasks, with just a few labeled examples.
  arXiv  Detail & Related papers  (2023-11-16T18:55:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.