Rethinking Structure Preservation in Text-Guided Image Editing with Visual Autoregressive Models

• URL: http://arxiv.org/abs/2603.28367v1
• Date: Mon, 30 Mar 2026 12:35:33 GMT
• Title: Rethinking Structure Preservation in Text-Guided Image Editing with Visual Autoregressive Models
• Abstract summary: We propose a novel text-guided image editing framework rooted in an analysis of intermediate feature distributions within visual autoregressive ( VAR) models.<n>First, we introduce a coarse-to-fine token localization strategy that can refine editable regions, balancing editing fidelity and background preservation.<n>Second, we analyze the intermediate representations of VAR models and identify structure-related features, by which we design a simple yet effective feature injection mechanism.<n>Third, we develop a reinforcement learning-based adaptive feature injection scheme that automatically learns scale- and layer-specific injection ratios to jointly optimize editing fidelity and structure preservation.
• Score: 18.626758540572894
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Visual autoregressive (VAR) models have recently emerged as a promising family of generative models, enabling a wide range of downstream vision tasks such as text-guided image editing. By shifting the editing paradigm from noise manipulation in diffusion-based methods to token-level operations, VAR-based approaches achieve better background preservation and significantly faster inference. However, existing VAR-based editing methods still face two key challenges: accurately localizing editable tokens and maintaining structural consistency in the edited results. In this work, we propose a novel text-guided image editing framework rooted in an analysis of intermediate feature distributions within VAR models. First, we introduce a coarse-to-fine token localization strategy that can refine editable regions, balancing editing fidelity and background preservation. Second, we analyze the intermediate representations of VAR models and identify structure-related features, by which we design a simple yet effective feature injection mechanism to enhance structural consistency between the edited and source images. Third, we develop a reinforcement learning-based adaptive feature injection scheme that automatically learns scale- and layer-specific injection ratios to jointly optimize editing fidelity and structure preservation. Extensive experiments demonstrate that our method achieves superior structural consistency and editing quality compared with state-of-the-art approaches, across both local and global editing scenarios.

Related papers

• Edge-Aware Image Manipulation via Diffusion Models with a Novel Structure-Preservation Loss [32.26030534230571]
  We propose a novel Structure Preservation Loss (SPL) to quantify structural differences between input and edited images.<n>Our training-free approach integrates SPL directly into the diffusion model's generative process to ensure structural fidelity.<n> Experiments confirm SPL enhances structural fidelity, delivering state-of-the-art performance in latent-diffusion-based image editing.
  arXiv  Detail & Related papers  (2026-01-23T11:06:51Z)
• LORE: Latent Optimization for Precise Semantic Control in Rectified Flow-based Image Editing [0.276240219662896]
  We introduce LORE, a training-free and efficient image editing method.<n>LORE directly optimize the inverted noise, addressing the core limitations in generalization and controllability of existing approaches.<n> Experimental results show that LORE significantly outperforms strong baselines in terms of semantic alignment, image quality, and background fidelity.
  arXiv  Detail & Related papers  (2025-08-05T06:45:04Z)
• Inverse-and-Edit: Effective and Fast Image Editing by Cycle Consistency Models [1.9389881806157316]
  In this work, we propose a novel framework that enhances image inversion using consistency models.<n>Our method introduces a cycle-consistency optimization strategy that significantly improves reconstruction accuracy.<n>We achieve state-of-the-art performance across various image editing tasks and datasets.
  arXiv  Detail & Related papers  (2025-06-23T20:34:43Z)
• Image Editing As Programs with Diffusion Models [69.05164729625052]
  We introduce Image Editing As Programs (IEAP), a unified image editing framework built upon the Diffusion Transformer (DiT) architecture.<n>IEAP approaches instructional editing through a reductionist lens, decomposing complex editing instructions into sequences of atomic operations.<n>Our framework delivers superior accuracy and semantic fidelity, particularly for complex, multi-step instructions.
  arXiv  Detail & Related papers  (2025-06-04T16:57:24Z)
• Anchor Token Matching: Implicit Structure Locking for Training-free AR Image Editing [60.102602955261084]
  Implicit Structure Locking (ISLock) is the first training-free editing strategy for AR visual models.<n>Our method preserves structural blueprints by dynamically aligning self-attention patterns with reference images.<n>Our findings pioneer the way for efficient and flexible AR-based image editing, further bridging the performance gap between diffusion and autoregressive generative models.
  arXiv  Detail & Related papers  (2025-04-14T17:25:19Z)
• Training-Free Text-Guided Image Editing with Visual Autoregressive Model [46.201510044410995]
  We propose a novel text-guided image editing framework based on Visual AutoRegressive modeling.<n>Our method eliminates the need for explicit inversion while ensuring precise and controlled modifications.<n>Our framework operates in a training-free manner and achieves high-fidelity editing with faster inference speeds.
  arXiv  Detail & Related papers  (2025-03-31T09:46:56Z)
• DCEdit: Dual-Level Controlled Image Editing via Precisely Localized Semantics [71.78350994830885]
  We present a novel approach to improving text-guided image editing using diffusion-based models.<n>Our method uses visual and textual self-attention to enhance the cross-attention map, which can serve as a regional cues to improve editing performance.<n>To fully compare our methods with other DiT-based approaches, we construct the RW-800 benchmark, featuring high resolution images, long descriptive texts, real-world images, and a new text editing task.
  arXiv  Detail & Related papers  (2025-03-21T02:14:03Z)
• PartEdit: Fine-Grained Image Editing using Pre-Trained Diffusion Models [80.98455219375862]
  We present the first text-based image editing approach for object parts based on pre-trained diffusion models.<n>Our approach is preferred by users 66-90% of the time in conducted user studies.
  arXiv  Detail & Related papers  (2025-02-06T13:08:43Z)
• Uniform Attention Maps: Boosting Image Fidelity in Reconstruction and Editing [66.48853049746123]
  We analyze reconstruction from a structural perspective and propose a novel approach that replaces traditional cross-attention with uniform attention maps.<n>Our method effectively minimizes distortions caused by varying text conditions during noise prediction.<n> Experimental results demonstrate that our approach not only excels in achieving high-fidelity image reconstruction but also performs robustly in real image composition and editing scenarios.
  arXiv  Detail & Related papers  (2024-11-29T12:11:28Z)
• Task-Oriented Diffusion Inversion for High-Fidelity Text-based Editing [60.730661748555214]
  We introduce textbfTask-textbfOriented textbfDiffusion textbfInversion (textbfTODInv), a novel framework that inverts and edits real images tailored to specific editing tasks. ToDInv seamlessly integrates inversion and editing through reciprocal optimization, ensuring both high fidelity and precise editability.
  arXiv  Detail & Related papers  (2024-08-23T22:16:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.