Neighbor-Aware Localized Concept Erasure in Text-to-Image Diffusion Models

• URL: http://arxiv.org/abs/2603.25994v1
• Date: Fri, 27 Mar 2026 00:58:11 GMT
• Title: Neighbor-Aware Localized Concept Erasure in Text-to-Image Diffusion Models
• Abstract summary: Concept erasure in text-to-image diffusion models seeks to remove undesired concepts while preserving overall generative capability.<n>We propose Neighbor-Aware Localized Concept Erasure (NLCE), a training-free framework designed to better preserve neighboring concepts while removing target concepts.
• Score: 11.53473951431438
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Concept erasure in text-to-image diffusion models seeks to remove undesired concepts while preserving overall generative capability. Localized erasure methods aim to restrict edits to the spatial region occupied by the target concept. However, we observe that suppressing a concept can unintentionally weaken semantically related neighbor concepts, reducing fidelity in fine-grained domains. We propose Neighbor-Aware Localized Concept Erasure (NLCE), a training-free framework designed to better preserve neighboring concepts while removing target concepts. It operates in three stages: (1) a spectrally-weighted embedding modulation that attenuates target concept directions while stabilizing neighbor concept representations, (2) an attention-guided spatial gate that identifies regions exhibiting residual concept activation, and (3) a spatially-gated hard erasure that eliminates remaining traces only where necessary. This neighbor-aware pipeline enables localized concept removal while maintaining the surrounding concept neighborhood structure. Experiments on fine-grained datasets (Oxford Flowers, Stanford Dogs) show that our method effectively removes target concepts while better preserving closely related categories. Additional results on celebrity identity, explicit content and artistic style demonstrate robustness and generalization to broader erasure scenarios.

Related papers

• EraseSAE: Surgical Concept Erasure in Text-to-Video Diffusion Models via Sparse Autoencoders [103.46709005379591]
  Concept erasure offers a principled remedy by removing unwanted semantics from pretrained models.<n>Existing approaches typically operate at a coarse quality, misaligned with the finegrained, distributed nature of concept representations.<n>We propose EraseSAE, a novel framework that leverages sparse autoencoders to achieve surgical concept erasure.
  arXiv  Detail & Related papers  (2026-09-03T10:23:37Z)
• GrOCE:Graph-Guided Online Concept Erasure for Text-to-Image Diffusion Models [24.278300091974085]
  Concept erasure aims to remove harmful, inappropriate, or copyrighted content from text-to-image diffusion models.<n>We propose Graph-Guided Online Concept Erasure (GrOCE), a training-free framework that performs precise and adaptive concept removal.
  arXiv  Detail & Related papers  (2025-11-17T04:47:16Z)
• SAGE: Exploring the Boundaries of Unsafe Concept Domain with Semantic-Augment Erasing [65.82241040239452]
  Concept erasing finetunes weights to unlearn undesirable concepts.<n>Existing methods treat unsafe concept as a fixed word and repeatedly erase it.<n>We introduce semantic-augment erasing which transforms concept word erasure into concept domain erasure.
  arXiv  Detail & Related papers  (2025-06-11T03:21:24Z)
• ACE: Attentional Concept Erasure in Diffusion Models [0.0]
  Attentional Concept Erasure integrates a closed-form attention manipulation with lightweight fine-tuning.<n>We show that ACE achieves state-of-the-art concept removal efficacy and robustness.<n>Compared to prior methods, ACE better balances generality (erasing concept and related terms) and specificity (preserving unrelated content)
  arXiv  Detail & Related papers  (2025-04-16T08:16:28Z)
• Fine-Grained Erasure in Text-to-Image Diffusion-based Foundation Models [56.35484513848296]
  FADE (Fine grained Attenuation for Diffusion Erasure) is an adjacency-aware unlearning algorithm for text-to-image generative models.<n>It removes target concepts with minimal impact on correlated concepts, achieving a 12% improvement in retention performance over state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-03-25T15:49:48Z)
• Localized Concept Erasure for Text-to-Image Diffusion Models Using Training-Free Gated Low-Rank Adaptation [12.098673137752861]
  Fine-tuning based concept erasing has demonstrated promising results in preventing generation of harmful contents from text-to-image diffusion models.<n>We introduce a framework called localized concept erasure, which allows for the deletion of only the specific area containing the target concept in the image.<n>We propose a training-free approach, dubbed Gated Low-rank adaptation for Concept Erasure (GLoCE), that injects a lightweight module into the diffusion model.
  arXiv  Detail & Related papers  (2025-03-16T04:53:20Z)
• Fantastic Targets for Concept Erasure in Diffusion Models and Where To Find Them [21.386640828092524]
  Concept erasure has emerged as a promising technique for mitigating the risk of harmful content generation in diffusion models.<n>We propose the Adaptive Guided Erasure (AGE) method, which emphdynamically selects optimal target concepts tailored to each undesirable concept.<n>Results show that AGE significantly outperforms state-of-the-art erasure methods on preserving unrelated concepts while maintaining effective erasure performance.
  arXiv  Detail & Related papers  (2025-01-31T08:17:23Z)
• OmniPrism: Learning Disentangled Visual Concept for Image Generation [57.21097864811521]
  Creative visual concept generation often draws inspiration from specific concepts in a reference image to produce relevant outcomes.<n>We propose OmniPrism, a visual concept disentangling approach for creative image generation.<n>Our method learns disentangled concept representations guided by natural language and trains a diffusion model to incorporate these concepts.
  arXiv  Detail & Related papers  (2024-12-16T18:59:52Z)
• Concept Replacer: Replacing Sensitive Concepts in Diffusion Models via Precision Localization [48.20360860166279]
  Large-scale diffusion models produce high-quality images but often generate unwanted content, such as sexually explicit or violent content.<n>We propose a novel approach for targeted concept replacing in diffusion models, enabling specific concepts to be removed without affecting non-target areas.<n>Our method introduces a dedicated concept localizer for precisely identifying the target concept during the denoising process, trained with few-shot learning to require minimal labeled data.<n>Within the identified region, we introduce a training-free Dual Prompts Cross-Attention (DPCA) module to substitute the target concept, ensuring minimal disruption to surrounding content.
  arXiv  Detail & Related papers  (2024-12-02T08:05:39Z)
• RealEra: Semantic-level Concept Erasure via Neighbor-Concept Mining [25.769144703607214]
  Concept erasure has been proposed to remove the model's knowledge about protected and inappropriate concepts. We propose RealEra to address this "concept residue" issue. We show that RealEra outperforms previous concept erasing methods in terms of superior erasing efficacy, specificity, and generality.
  arXiv  Detail & Related papers  (2024-10-11T17:55:30Z)
• Implicit Concept Removal of Diffusion Models [92.55152501707995]
  Text-to-image (T2I) diffusion models often inadvertently generate unwanted concepts such as watermarks and unsafe images. We present the Geom-Erasing, a novel concept removal method based on the geometric-driven control.
  arXiv  Detail & Related papers  (2023-10-09T17:13:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.