Erase Diffusion: Empowering Object Removal Through Calibrating Diffusion Pathways

• URL: http://arxiv.org/abs/2503.07026v1
• Date: Mon, 10 Mar 2025 08:06:51 GMT
• Title: Erase Diffusion: Empowering Object Removal Through Calibrating Diffusion Pathways
• Authors: Yi Liu, Hao Zhou, Wenxiang Shang, Ran Lin, Benlei Cui,
• Abstract summary: Erase inpainting aims to precisely remove target objects within masked regions while preserving the overall consistency of the surrounding content.<n>We propose a novel Erase Diffusion, termed EraDiff, aimed at unleashing the potential power of standard diffusion in the context of object removal.<n>Our proposed EraDiff achieves state-of-the-art performance on the OpenImages V5 dataset and demonstrates significant superiority in real-world scenarios.
• Score: 13.08168394252538
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Erase inpainting, or object removal, aims to precisely remove target objects within masked regions while preserving the overall consistency of the surrounding content. Despite diffusion-based methods have made significant strides in the field of image inpainting, challenges remain regarding the emergence of unexpected objects or artifacts. We assert that the inexact diffusion pathways established by existing standard optimization paradigms constrain the efficacy of object removal. To tackle these challenges, we propose a novel Erase Diffusion, termed EraDiff, aimed at unleashing the potential power of standard diffusion in the context of object removal. In contrast to standard diffusion, the EraDiff adapts both the optimization paradigm and the network to improve the coherence and elimination of the erasure results. We first introduce a Chain-Rectifying Optimization (CRO) paradigm, a sophisticated diffusion process specifically designed to align with the objectives of erasure. This paradigm establishes innovative diffusion transition pathways that simulate the gradual elimination of objects during optimization, allowing the model to accurately capture the intent of object removal. Furthermore, to mitigate deviations caused by artifacts during the sampling pathways, we develop a simple yet effective Self-Rectifying Attention (SRA) mechanism. The SRA calibrates the sampling pathways by altering self-attention activation, allowing the model to effectively bypass artifacts while further enhancing the coherence of the generated content. With this design, our proposed EraDiff achieves state-of-the-art performance on the OpenImages V5 dataset and demonstrates significant superiority in real-world scenarios.

Related papers

• A Diffusion-Based Framework for Occluded Object Movement [39.6345172890042]
  We propose a Diffusion-based framework specifically designed for Occluded Object Movement, named DiffOOM. The de-occlusion branch utilizes a background color-fill strategy and a continuously updated object mask to focus the diffusion process on completing the obscured portion of the target object. Concurrently, the movement branch employs latent optimization to place the completed object in the target location and adopts local text-conditioned guidance to integrate the object into new surroundings appropriately.
  arXiv  Detail & Related papers  (2025-04-02T16:29:30Z)
• One-for-More: Continual Diffusion Model for Anomaly Detection [61.12622458367425]
  Anomaly detection methods utilize diffusion models to generate or reconstruct normal samples when given arbitrary anomaly images.<n>Our study found that the diffusion model suffers from severe faithfulness hallucination'' and catastrophic forgetting''<n>We propose a continual diffusion model that uses gradient projection to achieve stable continual learning.
  arXiv  Detail & Related papers  (2025-02-27T07:47:27Z)
• Attentive Eraser: Unleashing Diffusion Model's Object Removal Potential via Self-Attention Redirection Guidance [4.295971864740951]
  Attentive Eraser is a tuning-free method to empower pre-trained diffusion models for stable and effective object removal.<n>We introduce Attention Activation and Suppression (ASS), which re-engineers the self-attention mechanism.<n>We also introduce Self-Attention Redirection Guidance (SARG), which utilizes the self-attention redirected by ASS to guide the generation process.
  arXiv  Detail & Related papers  (2024-12-17T14:56:59Z)
• CLIPAway: Harmonizing Focused Embeddings for Removing Objects via Diffusion Models [16.58831310165623]
  CLIPAway is a novel approach leveraging CLIP embeddings to focus on background regions while excluding foreground elements. It enhances inpainting accuracy and quality by identifying embeddings that prioritize the background. Unlike other methods that rely on specialized training datasets or costly manual annotations, CLIPAway provides a flexible, plug-and-play solution.
  arXiv  Detail & Related papers  (2024-06-13T17:50:28Z)
• Erasing Undesirable Influence in Diffusion Models [51.225365010401006]
  Diffusion models are highly effective at generating high-quality images but pose risks, such as the unintentional generation of NSFW (not safe for work) content. In this work, we introduce EraseDiff, an algorithm designed to preserve the utility of the diffusion model on retained data while removing the unwanted information associated with the data to be forgotten.
  arXiv  Detail & Related papers  (2024-01-11T09:30:36Z)
• D-SCo: Dual-Stream Conditional Diffusion for Monocular Hand-Held Object Reconstruction [74.49121940466675]
  We introduce centroid-fixed dual-stream conditional diffusion for monocular hand-held object reconstruction. First, to avoid the object centroid from deviating, we utilize a novel hand-constrained centroid fixing paradigm. Second, we introduce a dual-stream denoiser to semantically and geometrically model hand-object interactions.
  arXiv  Detail & Related papers  (2023-11-23T20:14:50Z)
• Stable Diffusion For Aerial Object Detection [4.014524824655107]
  We introduce a synthetic data augmentation framework tailored for aerial images. It encompasses sparse-to-dense region of interest (ROI) extraction to bridge the semantic gap, fine-tuning the diffusion model with low-rank adaptation (LORA) to circumvent exhaustive retraining, and finally, a Copy-Paste method to compose synthesized objects with backgrounds.
  arXiv  Detail & Related papers  (2023-11-21T04:38:21Z)
• Global Structure-Aware Diffusion Process for Low-Light Image Enhancement [64.69154776202694]
  This paper studies a diffusion-based framework to address the low-light image enhancement problem. We advocate for the regularization of its inherent ODE-trajectory. Experimental evaluations reveal that the proposed framework attains distinguished performance in low-light enhancement.
  arXiv  Detail & Related papers  (2023-10-26T17:01:52Z)
• Hierarchical Integration Diffusion Model for Realistic Image Deblurring [71.76410266003917]
  Diffusion models (DMs) have been introduced in image deblurring and exhibited promising performance. We propose the Hierarchical Integration Diffusion Model (HI-Diff), for realistic image deblurring. Experiments on synthetic and real-world blur datasets demonstrate that our HI-Diff outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-05-22T12:18:20Z)
• ShadowDiffusion: When Degradation Prior Meets Diffusion Model for Shadow Removal [74.86415440438051]
  We propose a unified diffusion framework that integrates both the image and degradation priors for highly effective shadow removal. Our model achieves a significant improvement in terms of PSNR, increasing from 31.69dB to 34.73dB over SRD dataset.
  arXiv  Detail & Related papers  (2022-12-09T07:48:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.