Semantically Aligned Gradient-Driven Context-Preserving Image Editing

• URL: http://arxiv.org/abs/2609.12691v1
• Date: Fri, 11 Sep 2026 10:37:32 GMT
• Title: Semantically Aligned Gradient-Driven Context-Preserving Image Editing
• Abstract summary: We present IABEdit, a model-agnostic framework that embeds differentiable semantic verification into training.<n>IABEdit is compatible with diverse backbones, including U-Net (Stable Diffusion) and MMDiT (FLUX)
• Score: 32.85393969503569
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Instruction-guided image editing has a training-time blind spot. Generative editors are never required to semantically verify whether their outputs actually satisfy the instruction. Supervision stops at reconstruction and input textual-level conditioning. This produces incomplete edits, spatial spillover, and poor localization. We present IABEdit, a model-agnostic framework that embeds differentiable semantic verification into training. A frozen vision-language model extracts spatially-aware descriptors from the ground-truth edit. A trainable aligner then reproduces them from the generated output. The residual between the two becomes a gradient that teaches the generator both what to edit and where, with no inference-time VLM cost. IABEdit is compatible with diverse backbones, including U-Net (Stable Diffusion) and MMDiT (FLUX), without altering their inference pipelines. On MagicBrush, it improves structural fidelity by +3.49 DINO-I over the best diffusion baseline and +1.26 over the best overall baseline, while remaining competitive on instruction alignment. It also achieves state-of-the-art instruction adherence performance on RealEdit and EMU Edit benchmarks based on embedding-based metrics. Most consequentially, on the D-LORD surveillance benchmark, it surpasses the proprietary Gemini agent by +5.13 DINO-P under heavy occlusion, where preserving identity is hardest. This shows that gradient-aligned VLM distillation holds up under real-world-like surveillance and occlusion conditions. Human and GPT-4o evaluations confirm perceptually precise, well-localized edits.

Related papers

• Multi-History-Step SDE Inversion for Image Editing with Superior Regional Awareness [48.14550823309452]
  MIEdit is a training-free editing framework based on SDE inversion.<n>MIEdit introduces a predictor-corrector multi-history-step scheme to achieve superior editing quality with fewer steps.
  arXiv  Detail & Related papers  (2026-09-06T13:37:46Z)
• Reasoning to Align: Implicit Reasoning in Diffusion Transformers for Video Editing [55.211537893248675]
  We propose RVEDiT, an implicit Reasoning Video Editing DiT framework built around two complementary components.<n> RVEDiT consistently outperforms state-of-the-art baselines, with particularly strong gains on localized and compositional edits.
  arXiv  Detail & Related papers  (2026-05-23T17:22:14Z)
• CoEditor++: Instruction-based Visual Editing via Cognitive Reasoning [98.98349220451216]
  CoEditor++ is a training-free framework that decomposes editing into "what to edit" and "how to edit"<n>We show that CoEditor++ achieves state-of-the-art performance in both general editing and responsible editing tasks.<n>Our findings suggest the potential toward cognitive-centric instruction-based image editing.
  arXiv  Detail & Related papers  (2026-01-31T12:20:46Z)
• RemEdit: Efficient Diffusion Editing with Riemannian Geometry [1.8594036119086927]
  RemEdit is a diffusion-based framework for image editing.<n>For editing fidelity, we use a mamba-based module and a goal-aware prompt enrichment pass from a Vision-Language Model.<n>For additional acceleration, we introduce a novel task-specific attention pruning mechanism.<n>RemEdit surpasses prior state-of-the-art editing frameworks while maintaining real-time performance under 50% pruning.
  arXiv  Detail & Related papers  (2026-01-25T17:58:57Z)
• Visual Autoregressive Modeling for Instruction-Guided Image Editing [97.04821896251681]
  We present a visual autoregressive framework that reframes image editing as a next-scale prediction problem.<n>VarEdit generates multi-scale target features to achieve precise edits.<n>It completes a $512times512$ editing in 1.2 seconds, making it 2.2$times$ faster than the similarly sized UltraEdit.
  arXiv  Detail & Related papers  (2025-08-21T17:59:32Z)
• UniEdit-I: Training-free Image Editing for Unified VLM via Iterative Understanding, Editing and Verifying [64.5307229755533]
  We introduce a novel training-free framework named UniEdit-I to enable the unified VLM with image editing capability.<n>We implement our method based on the latest BLIP3-o and achieved state-of-the-art (SOTA) performance on the GEdit-Bench benchmark.
  arXiv  Detail & Related papers  (2025-08-05T06:42:09Z)
• FireEdit: Fine-grained Instruction-based Image Editing via Region-aware Vision Language Model [54.693572837423226]
  FireEdit is an innovative Fine-grained Instruction-based image editing framework that exploits a REgion-aware VLM.<n>FireEdit is designed to accurately comprehend user instructions and ensure effective control over the editing process.<n>Our approach surpasses the state-of-the-art instruction-based image editing methods.
  arXiv  Detail & Related papers  (2025-03-25T16:59:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.