Making Image Editing Easier via Adaptive Task Reformulation with Agentic Executions

• URL: http://arxiv.org/abs/2604.15917v1
• Date: Fri, 17 Apr 2026 10:17:22 GMT
• Title: Making Image Editing Easier via Adaptive Task Reformulation with Agentic Executions
• Abstract summary: We propose an adaptive task reformulation framework that improves editing performance without modifying the underlying model.<n>Our key idea is to transform the original image-instruction pair into a sequence of operations that are dynamically determined and executed by a MLLM agent.
• Score: 27.891822809255682
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Instruction guided image editing has advanced substantially with recent generative models, yet it still fails to produce reliable results across many seemingly simple cases. We observe that a large portion of these failures stem not from insufficient model capacity, but from poorly formulated editing tasks, such as those involving small targets, implicit spatial relations, or under-specified instructions. In this work, we frame image editing failures as a task formulation problem and propose an adaptive task reformulation framework that improves editing performance without modifying the underlying model. Our key idea is to transform the original image-instruction pair into a sequence of operations that are dynamically determined and executed by a MLLM agent through analysis, routing, reformulation, and feedback-driven refinement. Experiments on multiple benchmarks, including ImgEdit, PICA, and RePlan, across diverse editing backbones such as Qwen Image Edit and Nano Banana, show consistent improvements, with especially large gains on challenging cases. These results suggest that task reformulation is a critical but underexplored factor, and that substantial gains can be achieved by better matching editing tasks to the effective operating regime of existing models.

Related papers

• Rethinking Scribble-Guided Image Editing: Generalization, Instruction Adherence, and Multi-Tasking [14.173267480504842]
  Scribble-guided image editing allows users to combine simple scribble annotations with text prompts to specify both where and how an image should be edited.<n>Existing models still exhibit unstable performance under this paradigm, especially in multi-task scenarios.<n>We conduct empirical studies using an open-source editing model and reveal an asymmetry in generalization.
  arXiv  Detail & Related papers  (2026-05-25T08:20:23Z)
• CAMEO: A Conditional and Quality-Aware Multi-Agent Image Editing Orchestrator [8.901958956012072]
  Conditional image editing is crucial in scenarios requiring strict structural control.<n>Most approaches rely on single-step generation.<n>We propose textbfCAMEO, a structured multi-agent framework that reformulates conditional editing as a quality-aware, feedback-driven process.
  arXiv  Detail & Related papers  (2026-04-03T16:27:02Z)
• Image Editing As Programs with Diffusion Models [69.05164729625052]
  We introduce Image Editing As Programs (IEAP), a unified image editing framework built upon the Diffusion Transformer (DiT) architecture.<n>IEAP approaches instructional editing through a reductionist lens, decomposing complex editing instructions into sequences of atomic operations.<n>Our framework delivers superior accuracy and semantic fidelity, particularly for complex, multi-step instructions.
  arXiv  Detail & Related papers  (2025-06-04T16:57:24Z)
• SPICE: A Synergistic, Precise, Iterative, and Customizable Image Editing Workflow [13.815228931600236]
  We introduce SPICE, a training-free workflow that accepts arbitrary resolutions and aspect ratios, accurately follows user requirements, and consistently improves image quality during more than 100 editing steps.<n>On a challenging realistic image-editing dataset, SPICE quantitatively outperforms state-of-the-art baselines and is consistently preferred by human annotators.
  arXiv  Detail & Related papers  (2025-04-13T19:13:04Z)
• EditAR: Unified Conditional Generation with Autoregressive Models [58.093860528672735]
  We propose EditAR, a single unified autoregressive framework for a variety of conditional image generation tasks.<n>The model takes both images and instructions as inputs, and predicts the edited images tokens in a vanilla next-token paradigm.<n>We evaluate its effectiveness across diverse tasks on established benchmarks, showing competitive performance to various state-of-the-art task-specific methods.
  arXiv  Detail & Related papers  (2025-01-08T18:59:35Z)
• Task-Oriented Diffusion Inversion for High-Fidelity Text-based Editing [60.730661748555214]
  We introduce textbfTask-textbfOriented textbfDiffusion textbfInversion (textbfTODInv), a novel framework that inverts and edits real images tailored to specific editing tasks. ToDInv seamlessly integrates inversion and editing through reciprocal optimization, ensuring both high fidelity and precise editability.
  arXiv  Detail & Related papers  (2024-08-23T22:16:34Z)
• FunEditor: Achieving Complex Image Edits via Function Aggregation with Diffusion Models [15.509233098264513]
  Diffusion models have demonstrated outstanding performance in generative tasks, making them ideal candidates for image editing.<n>We introduce FunEditor, an efficient diffusion model designed to learn atomic editing functions and perform complex edits by aggregating simpler functions.<n>With only 4 steps of inference, FunEditor achieves 5-24x inference speedups over existing popular methods.
  arXiv  Detail & Related papers  (2024-08-16T02:33:55Z)
• The Butterfly Effect of Model Editing: Few Edits Can Trigger Large Language Models Collapse [58.0132400208411]
  Even a single edit can trigger model collapse, manifesting as significant performance degradation in various benchmark tasks. benchmarking Large Language Models after each edit is impractically time-consuming and resource-intensive. We have utilized GPT-3.5 to develop a new dataset, HardEdit, based on hard cases.
  arXiv  Detail & Related papers  (2024-02-15T01:50:38Z)
• Tuning-Free Inversion-Enhanced Control for Consistent Image Editing [44.311286151669464]
  We present a novel approach called Tuning-free Inversion-enhanced Control (TIC) TIC correlates features from the inversion process with those from the sampling process to mitigate the inconsistency in DDIM reconstruction. We also propose a mask-guided attention concatenation strategy that combines contents from both the inversion and the naive DDIM editing processes.
  arXiv  Detail & Related papers  (2023-12-22T11:13:22Z)
• Emu Edit: Precise Image Editing via Recognition and Generation Tasks [62.95717180730946]
  We present Emu Edit, a multi-task image editing model which sets state-of-the-art results in instruction-based image editing. We train it to multi-task across an unprecedented range of tasks, such as region-based editing, free-form editing, and Computer Vision tasks. We show that Emu Edit can generalize to new tasks, such as image inpainting, super-resolution, and compositions of editing tasks, with just a few labeled examples.
  arXiv  Detail & Related papers  (2023-11-16T18:55:58Z)
• End-to-End Visual Editing with a Generatively Pre-Trained Artist [78.5922562526874]
  We consider the targeted image editing problem: blending a region in a source image with a driver image that specifies the desired change. We propose a self-supervised approach that simulates edits by augmenting off-the-shelf images in a target domain. We show that different blending effects can be learned by an intuitive control of the augmentation process, with no other changes required to the model architecture.
  arXiv  Detail & Related papers  (2022-05-03T17:59:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.