Edit Where You Mean: Region-Aware Adapter Injection for Mask-Free Local Image Editing

• URL: http://arxiv.org/abs/2604.23763v2
• Date: Thu, 30 Apr 2026 04:05:16 GMT
• Title: Edit Where You Mean: Region-Aware Adapter Injection for Mask-Free Local Image Editing
• Abstract summary: Large diffusion transformers (DiTs) follow global editing instructions well but consistently leak local edits into unrelated regions.<n>We introduce AdaptEdit, a co-trained, instruction- and region-aware adapter framework.<n> AdaptEdit achieves state-of-the-art results, simultaneously outperforming mask-free and oracle-mask baselines.
• Score: 28.295031615161136
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Large diffusion transformers (DiTs) follow global editing instructions well but consistently leak local edits into unrelated regions, because joint-attention architectures offer no explicit channel telling the network where to apply the edit. We introduce AdaptEdit, a co-trained, instruction- and region-aware adapter framework that retro-fits a frozen DiT into a precise local editor without modifying its backbone weights. A lightweight Block Adapter at every transformer block injects a structured condition stream that factorizes what to edit (instruction semantics) from where to edit (spatial mask); a learned SpatialGate routes the adapter signal selectively into the edit region while keeping the rest of the image near-identical to the source; and a Region-Aware Loss focuses the training objective on the changing pixels. Because these components make the backbone's internal representation mask-aware end-to-end, a thin MaskPredictor head trained jointly with the editor can ground the edit region directly from the instruction and source image -- eliminating any user-mask requirement at deployment. We evaluate on two complementary benchmarks: MagicBrush (paired ground-truth targets) to measure pixel-level preservation and edit accuracy, and Emu-Edit Test (no ground-truth images, 9 diverse edit categories) to stress-test instruction following and generalization across edit types. On both, AdaptEdit achieves state-of-the-art results, simultaneously outperforming mask-free and oracle-mask baselines. A seven-variant ablation cleanly isolates the contribution of each component.

Related papers

• PC-Edit: Prompt-Contrastive Region Discovery and Region-Guided Editing [6.411145731142763]
  PC-Edit is a prompt-contrastive framework for training-free MM-DiT editing.<n>We show that PC-Edit achieves the best editing quality and background preservation among methods without user-specified edit regions.
  arXiv  Detail & Related papers  (2026-07-23T13:42:03Z)
• WhereEdit: Mask-aware Local Latent Editing for One-Step Image Editing [16.758428220880734]
  One-step text-to-image (T2I) models enable efficient image synthesis and provide new opportunities for real-time image editing.<n>Existing one-step editing methods rely on text conditioning for semantic transformation, lacking explicit spatial control over textitwhere to edit.<n>We propose textbfWhereEdit, a framework that reformulates one-step editing as localized adaptive editing.
  arXiv  Detail & Related papers  (2026-07-23T03:11:25Z)
• SIGMA: Semantic-Difference Instruction-Grounding Mask Annotator for Text-Driven Image Manipulation Localization [64.6372217552272]
  Public editing datasets contain millions of structurally identical (original, edited) pairs to IML training samples, lacking only pixel-level masks.<n>We propose SIGMA, which performs semantic-feature differencing in a vision foundation backbone.<n>It produces a 1.1M IML training set that improves six diverse detectors by +18.34% F1 across five datasets.
  arXiv  Detail & Related papers  (2026-05-27T03:55:13Z)
• Masked Generative Transformer Is What You Need for Image Editing [91.7627902312967]
  Diffusion models entangle edited regions with surrounding context, causing modifications to propagate into areas that should remain intact.<n>We propose a fundamentally different approach by leveraging Masked Generative Transformers (MGTs), whose localized token-prediction paradigm naturally confines changes to intended regions.<n>We present EditMGT, an MGT-based editing framework that is the first of its kind.
  arXiv  Detail & Related papers  (2026-05-11T17:05:52Z)
• Rethinking Where to Edit: Task-Aware Localization for Instruction-Based Image Editing [30.23368654238746]
  We propose a training-free, task-aware edit localization framework for instruction-based image editing.<n>Our framework consistently improves non-edit region consistency while maintaining strong instruction-following performance.
  arXiv  Detail & Related papers  (2026-04-22T07:08:01Z)
• FusionEdit: Semantic Fusion and Attention Modulation for Training-Free Image Editing [7.53296048773288]
  Text-guided image editing aims to modify specific regions according to the target prompt while preserving the identity of the source image.<n>Recent methods exploit explicit binary masks to constrain editing, but hard mask boundaries introduce artifacts and reduce editability.<n>We propose FusionEdit, a training-free image editing framework that achieves precise and controllable edits.
  arXiv  Detail & Related papers  (2026-02-09T14:34:18Z)
• ProEdit: Inversion-based Editing From Prompts Done Right [63.554692704101]
  Inversion-based visual editing provides an effective and training-free way to edit an image or a video based on user instructions.<n>Existing methods typically inject source image information during the sampling process to maintain editing consistency.<n>We propose ProEdit to address this issue both in the attention and the latent aspects.
  arXiv  Detail & Related papers  (2025-12-26T18:59:14Z)
• SpotEdit: Selective Region Editing in Diffusion Transformers [66.44912649206553]
  SpotEdit is a training-free diffusion editing framework that selectively updates only the modified regions.<n>By reducing unnecessary computation and maintaining high fidelity in unmodified areas, SpotEdit achieves efficient and precise image editing.
  arXiv  Detail & Related papers  (2025-12-26T14:59:41Z)
• FlowDC: Flow-Based Decoupling-Decay for Complex Image Editing [52.54102743380658]
  We propose FlowDC, which decouples the complex editing into multiple sub-editing effects and superposes them in parallel during the editing process.<n>FlowDC shows superior results compared with existing methods.
  arXiv  Detail & Related papers  (2025-12-12T09:08:39Z)
• X-Edit: Detecting and Localizing Edits in Images Altered by Text-Guided Diffusion Models [3.610796534465868]
  Experimental results demonstrate that X-Edit accurately localizes edits in images altered by text-guided diffusion models.<n>This highlights X-Edit's potential as a robust forensic tool for detecting and pinpointing manipulations introduced by advanced image editing techniques.
  arXiv  Detail & Related papers  (2025-05-16T23:29:38Z)
• Addressing Text Embedding Leakage in Diffusion-based Image Editing [33.1686050396517]
  We introduce Attribute-Leakage-free Editing (ALE), a framework that tackles attribute leakage at its source.<n>ALE combines Object-Restricted Embeddings (ORE) to disentangle text embeddings, Region-Guided Blending for Cross-Attention Masking (RGB-CAM) for spatially precise attention, and Background Blending (BB) to preserve non-edited content.
  arXiv  Detail & Related papers  (2024-12-06T02:10:07Z)
• ZONE: Zero-Shot Instruction-Guided Local Editing [56.56213730578504]
  We propose a Zero-shot instructiON-guided local image Editing approach, termed ZONE. We first convert the editing intent from the user-provided instruction into specific image editing regions through InstructPix2Pix. We then propose a Region-IoU scheme for precise image layer extraction from an off-the-shelf segment model.
  arXiv  Detail & Related papers  (2023-12-28T02:54:34Z)
• Imagen Editor and EditBench: Advancing and Evaluating Text-Guided Image Inpainting [53.708523312636096]
  We present Imagen Editor, a cascaded diffusion model built, by fine-tuning on text-guided image inpainting. edits are faithful to the text prompts, which is accomplished by using object detectors to propose inpainting masks during training. To improve qualitative and quantitative evaluation, we introduce EditBench, a systematic benchmark for text-guided image inpainting.
  arXiv  Detail & Related papers  (2022-12-13T21:25:11Z)
• EditGAN: High-Precision Semantic Image Editing [120.49401527771067]
  EditGAN is a novel method for high quality, high precision semantic image editing. We show that EditGAN can manipulate images with an unprecedented level of detail and freedom. We can also easily combine multiple edits and perform plausible edits beyond EditGAN training data.
  arXiv  Detail & Related papers  (2021-11-04T22:36:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.