Aligned Stable Inpainting: Mitigating Unwanted Object Insertion and Preserving Color Consistency

• URL: http://arxiv.org/abs/2601.15368v2
• Date: Wed, 25 Feb 2026 16:19:43 GMT
• Title: Aligned Stable Inpainting: Mitigating Unwanted Object Insertion and Preserving Color Consistency
• Abstract summary: We introduce the framework of Aligned Stable inpainting with UnKnown Areas prior (ASUKA)<n>To reduce unwanted object insertion, we use reconstruction-based principled priors to guide the generative model.<n>To address color inconsistency, we design a specialized VAE decoder that formulates latent-to-image decoding as a local task.
• Score: 65.75215546331243
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Generative image inpainting can produce realistic, high-fidelity results even with large, irregular masks. However, existing methods still face key issues that make inpainted images look unnatural. In this paper, we identify two main problems: (1) Unwanted object insertion: generative models may hallucinate arbitrary objects in the masked region that do not match the surrounding context. (2) Color inconsistency: inpainted regions often exhibit noticeable color shifts, leading to smeared textures and degraded image quality. We analyze the underlying causes of these issues and propose efficient post-hoc solutions for pre-trained inpainting models. Specifically, we introduce the principled framework of Aligned Stable inpainting with UnKnown Areas prior (ASUKA). To reduce unwanted object insertion, we use reconstruction-based priors to guide the generative model, suppressing hallucinated objects while preserving generative flexibility. To address color inconsistency, we design a specialized VAE decoder that formulates latent-to-image decoding as a local harmonization task. This design significantly reduces color shifts and produces more color-consistent results. We implement ASUKA on two representative inpainting architectures: a U-Net-based model and a DiT-based model. We analyze and propose lightweight injection strategies that minimize interference with the model's original generation capacity while ensuring the mitigation of the two issues. We evaluate ASUKA using the Places2 dataset and MISATO, our proposed diverse benchmark. Experiments show that ASUKA effectively suppresses object hallucination and improves color consistency, outperforming standard diffusion, rectified flow models, and other inpainting methods. Dataset, models and codes will be released in github.

Related papers

• DACoN: DINO for Anime Paint Bucket Colorization with Any Number of Reference Images [0.0]
  DACoN is a framework that leverages foundation models to capture part-level semantics, even in line drawings.<n>Our method fuses low-resolution semantic features from foundation models with high-resolution spatial features from CNNs for fine-grained yet robust feature extraction.
  arXiv  Detail & Related papers  (2025-09-18T07:24:52Z)
• OmniPaint: Mastering Object-Oriented Editing via Disentangled Insertion-Removal Inpainting [54.525583840585305]
  We introduce OmniPaint, a unified framework that re-conceptualizes object removal and insertion as interdependent processes.<n>Our novel CFD metric offers a robust, reference-free evaluation of context consistency and object hallucination.
  arXiv  Detail & Related papers  (2025-03-11T17:55:27Z)
• Improving Text-guided Object Inpainting with Semantic Pre-inpainting [95.17396565347936]
  We decompose the typical single-stage object inpainting into two cascaded processes: semantic pre-inpainting and high-fieldity object generation. To achieve this, we cascade a Transformer-based semantic inpainter and an object inpainting diffusion model, leading to a novel CAscaded Transformer-Diffusion framework.
  arXiv  Detail & Related papers  (2024-09-12T17:55:37Z)
• Towards Enhanced Image Inpainting: Mitigating Unwanted Object Insertion and Preserving Color Consistency [78.0488707697235]
  Post-processing approach dubbed ASUKA (Aligned Stable inpainting with UnKnown Areas prior) to improve inpainting models.<n>Masked Auto-Encoder (MAE) for reconstruction-based priors mitigates object hallucination.<n> specialized VAE decoder that treats latent-to-image decoding as a local task.
  arXiv  Detail & Related papers  (2023-12-08T05:08:06Z)
• Perceptual Artifacts Localization for Inpainting [60.5659086595901]
  We propose a new learning task of automatic segmentation of inpainting perceptual artifacts. We train advanced segmentation networks on a dataset to reliably localize inpainting artifacts within inpainted images. We also propose a new evaluation metric called Perceptual Artifact Ratio (PAR), which is the ratio of objectionable inpainted regions to the entire inpainted area.
  arXiv  Detail & Related papers  (2022-08-05T18:50:51Z)
• Very Long Natural Scenery Image Prediction by Outpainting [96.8509015981031]
  Outpainting receives less attention due to two challenges in it. First challenge is how to keep the spatial and content consistency between generated images and original input. Second challenge is how to maintain high quality in generated results.
  arXiv  Detail & Related papers  (2019-12-29T16:29:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.