HarmonPaint: Harmonized Training-Free Diffusion Inpainting

• URL: http://arxiv.org/abs/2507.16732v1
• Date: Tue, 22 Jul 2025 16:14:35 GMT
• Title: HarmonPaint: Harmonized Training-Free Diffusion Inpainting
• Authors: Ying Li, Xinzhe Li, Yong Du, Yangyang Xu, Junyu Dong, Shengfeng He,
• Abstract summary: HarmonPaint is a training-free inpainting framework that seamlessly integrates with the attention mechanisms of diffusion models.<n>By leveraging masking strategies within self-attention, HarmonPaint ensures structural fidelity without model retraining or fine-tuning.
• Score: 58.870763247178495
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Existing inpainting methods often require extensive retraining or fine-tuning to integrate new content seamlessly, yet they struggle to maintain coherence in both structure and style between inpainted regions and the surrounding background. Motivated by these limitations, we introduce HarmonPaint, a training-free inpainting framework that seamlessly integrates with the attention mechanisms of diffusion models to achieve high-quality, harmonized image inpainting without any form of training. By leveraging masking strategies within self-attention, HarmonPaint ensures structural fidelity without model retraining or fine-tuning. Additionally, we exploit intrinsic diffusion model properties to transfer style information from unmasked to masked regions, achieving a harmonious integration of styles. Extensive experiments demonstrate the effectiveness of HarmonPaint across diverse scenes and styles, validating its versatility and performance.

Related papers

• GuidPaint: Class-Guided Image Inpainting with Diffusion Models [1.1902474395094222]
  We propose GuidPaint, a training-free, class-guided image inpainting framework.<n>We show that GuidPaint achieves clear improvements over existing context-aware inpainting methods in both qualitative and quantitative evaluations.
  arXiv  Detail & Related papers  (2025-07-29T09:36:52Z)
• Style Composition within Distinct LoRA modules for Traditional Art [21.954368353156546]
  We propose a zero-shot diffusion pipeline that naturally blends multiple styles.<n>We leverage the fact that lower-noise latents carry stronger stylistic information.<n>We incorporate depth-map conditioning via ControlNet into the diffusion framework.
  arXiv  Detail & Related papers  (2025-07-16T07:36:07Z)
• Towards Seamless Borders: A Method for Mitigating Inconsistencies in Image Inpainting and Outpainting [22.46566055053259]
  We propose two novel methods to address discrepancy issues in diffusion-based inpainting models.<n>First, we introduce a modified Variational Autoencoder that corrects color imbalances, ensuring that the final inpainted results are free of color mismatches.<n>Second, we propose a two-step training strategy that improves the blending of generated and existing image content during the diffusion process.
  arXiv  Detail & Related papers  (2025-06-14T15:02:56Z)
• From Missing Pieces to Masterpieces: Image Completion with Context-Adaptive Diffusion [98.31811240195324]
  ConFill is a novel framework that reduces discrepancies between generated and original images at each diffusion step.<n>It outperforms current methods, setting a new benchmark in image completion.
  arXiv  Detail & Related papers  (2025-04-19T13:40:46Z)
• Diffusion Models with Anisotropic Gaussian Splatting for Image Inpainting [0.0]
  We propose a novel inpainting method that combines diffusion models with anisotropic Gaussian splatting to capture both local structures and global context effectively.<n>Our method outperforms state-of-the-art techniques, producing visually plausible results with enhanced structural integrity and texture realism.
  arXiv  Detail & Related papers  (2024-12-02T16:29:06Z)
• ZePo: Zero-Shot Portrait Stylization with Faster Sampling [61.14140480095604]
  This paper presents an inversion-free portrait stylization framework based on diffusion models that accomplishes content and style feature fusion in merely four sampling steps. We propose a feature merging strategy to amalgamate redundant features in Consistency Features, thereby reducing the computational load of attention control.
  arXiv  Detail & Related papers  (2024-08-10T08:53:41Z)
• Coherent and Multi-modality Image Inpainting via Latent Space Optimization [61.99406669027195]
  PILOT (intextbfPainting vtextbfIa textbfLatent textbfOptextbfTimization) is an optimization approach grounded on a novel textitsemantic centralization and textitbackground preservation loss. Our method searches latent spaces capable of generating inpainted regions that exhibit high fidelity to user-provided prompts while maintaining coherence with the background.
  arXiv  Detail & Related papers  (2024-07-10T19:58:04Z)
• BrushNet: A Plug-and-Play Image Inpainting Model with Decomposed Dual-Branch Diffusion [61.90969199199739]
  BrushNet is a novel plug-and-play dual-branch model engineered to embed pixel-level masked image features into any pre-trained DM. BrushNet's superior performance over existing models across seven key metrics, including image quality, mask region preservation, and textual coherence.
  arXiv  Detail & Related papers  (2024-03-11T17:59:31Z)
• FreePIH: Training-Free Painterly Image Harmonization with Diffusion Model [19.170302996189335]
  Our FreePIH method tames the denoising process as a plug-in module for foreground image style transfer. We make use of multi-scale features to enforce the consistency of the content and stability of the foreground objects in the latent space. Our method can surpass representative baselines by large margins.
  arXiv  Detail & Related papers  (2023-11-25T04:23:49Z)
• Uni-paint: A Unified Framework for Multimodal Image Inpainting with Pretrained Diffusion Model [19.800236358666123]
  We propose Uni-paint, a unified framework for multimodal inpainting. Uni-paint offers various modes of guidance, including text-driven, stroke-driven, exemplar-driven inpainting. Our approach achieves comparable results to existing single-modal methods.
  arXiv  Detail & Related papers  (2023-10-11T06:11:42Z)
• Gradpaint: Gradient-Guided Inpainting with Diffusion Models [71.47496445507862]
  Denoising Diffusion Probabilistic Models (DDPMs) have recently achieved remarkable results in conditional and unconditional image generation. We present GradPaint, which steers the generation towards a globally coherent image. We generalizes well to diffusion models trained on various datasets, improving upon current state-of-the-art supervised and unsupervised methods.
  arXiv  Detail & Related papers  (2023-09-18T09:36:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.