GRACE: Adaptive Concept Erasure with Geometry-Guided Retention in Diffusion Models

• URL: http://arxiv.org/abs/2609.12731v1
• Date: Fri, 11 Sep 2026 11:29:37 GMT
• Title: GRACE: Adaptive Concept Erasure with Geometry-Guided Retention in Diffusion Models
• Abstract summary: GRACE is a structured concept erasure framework designed to enable localized and selective intervention.<n>We introduce a semantically weighted sensitive subspace estimation to precisely lock intervention directions.<n>We also introduce an energy-driven dynamic gating mechanism that adaptively controls the timing and strength of intervention at inference.
• Score: 16.95462517490593
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Text-to-image (T2I) diffusion models inevitably internalize sensitive or non-compliant concepts from large-scale pretraining data, necessitating post-hoc concept erasure. However, existing erasure methods often lack explicit constraints on parameter updates, leading to over-intervention and unintended semantic drift. In addition, many methods rely on manually crafted counterfactual supervision, such as surrogate prompts, which incurs substantial data construction costs that limit scalability to new concepts. To address these limitations, we propose GRACE, a structured concept erasure framework designed to enable localized and selective intervention. Specifically, we introduce a semantically weighted sensitive subspace estimation to precisely lock intervention directions, and employ lightweight subspace-constrained adapters to prevent global semantic disturbance. To eliminate the dependency on manual prompt engineering, we design an automatically decoupled safe-anchor mechanism. To mitigate semantic drift induced by excessive intervention, we introduce an energy-driven dynamic gating mechanism that adaptively controls the timing and strength of intervention at inference. Extensive experiments demonstrate that our method achieves a superior balance between erasure effectiveness and generation fidelity. Compared with the average performance of five state-of-the-art (SOTA) concept erasure methods, our method improves the fine-grained NSFW reduction rate by $17.86\%$, while reducing the macro-averaged target CLIP Score and preservation-oriented Fréchet Inception Distance (FID) by $4.75\%$ and $50.58\%$, respectively, indicating stronger concept suppression with substantially improved preservation of the original model's generative utility.

Related papers

• EraseSAE: Surgical Concept Erasure in Text-to-Video Diffusion Models via Sparse Autoencoders [103.46709005379591]
  Concept erasure offers a principled remedy by removing unwanted semantics from pretrained models.<n>Existing approaches typically operate at a coarse quality, misaligned with the finegrained, distributed nature of concept representations.<n>We propose EraseSAE, a novel framework that leverages sparse autoencoders to achieve surgical concept erasure.
  arXiv  Detail & Related papers  (2026-09-03T10:23:37Z)
• PEAK: Precise and Persistent Concept Erasure via k-Sparse Autoencoders [25.995968388013555]
  Erasing concepts from large-scale text-to-image (T2I) diffusion models has become increasingly crucial due to the growing concerns over copyright infringement, privacy violations, and offensive content.<n>We propose PEAK, a textbftextitprecise and textbftextitpersistent concept erasure framework via k-Sparse Autoencoders (kSAEs)
  arXiv  Detail & Related papers  (2026-08-11T14:38:01Z)
• IB-Flow: Information Bottleneck-Guided CFG Distillation for Few-Step Text-to-Image Generation [9.17007090517294]
  Few-step distillation targeting the Free Guidance trajectory has emerged as the prevalent dual-dimensional compression paradigm.<n>We re-examine the distillation procedure through the theoretical lens of Information Theory.<n>We propose an instance-aware selection mechanism that transmutes the intractable KL divergence constraint into a zero-overhead closed-form solution.
  arXiv  Detail & Related papers  (2026-07-10T06:43:43Z)
• SACE: Concept Erasure at the Semantic Singularity in Visual Autoregressive Models [10.865745057624833]
  We introduce the first scale-aware concept erasure framework (SACE) for visual autoregressive ( VAR) models.<n>By strictly confining interventions to the first scale, our approach couples an Entropy-Regularized Erasure Objective to prevent high-entropy sampling degeneration.
  arXiv  Detail & Related papers  (2026-06-14T13:39:00Z)
• FlowErase-RL: Rethinking Concept Erasure as Reward Optimization in Flow Matching Models [47.80885954469943]
  Flow matching models have significantly improved text-to-image generation quality, but also introduce growing safety risks.<n>We propose emphFlowErase-RL, the first GRPO-based framework for concept erasure in flow matching models.<n>We show that our method achieves state-of-the-art erasure performance while maintaining strong image quality and semantic alignment.
  arXiv  Detail & Related papers  (2026-05-19T12:10:09Z)
• Dynamic Eraser for Guided Concept Erasure in Diffusion Models [0.0]
  We present Dynamic Semantic Steering (DSS), a lightweight, training-free framework for interpretable and controllable concept erasure.<n> DSS achieves an average erasure rate of 91.0%, significantly outperforming SOTA methods (from 18.6% to 85.9%) with minimal impact on output fidelity.
  arXiv  Detail & Related papers  (2026-04-13T02:25:14Z)
• Bridging Semantic and Kinematic Conditions with Diffusion-based Discrete Motion Tokenizer [55.9892973179428]
  MoTok is a discrete motion tokenizer that decouples semantic abstraction from fine-grained reconstruction.<n>On HumanML3D, our method significantly improves controllability and fidelity over MaskControl while using only one-sixth of the tokens.
  arXiv  Detail & Related papers  (2026-03-19T17:59:51Z)
• ERIS: An Energy-Guided Feature Disentanglement Framework for Out-of-Distribution Time Series Classification [51.07970070817353]
  An ideal time series classification (TSC) should be able to capture invariant representations.<n>Current methods are largely unguided, lacking the semantic direction required to isolate truly universal features.<n>We propose an end-to-end Energy-Regularized Information for Shift-Robustness framework to enable guided and reliable feature disentanglement.
  arXiv  Detail & Related papers  (2025-08-19T12:13:41Z)
• CURE: Concept Unlearning via Orthogonal Representation Editing in Diffusion Models [7.68494752148263]
  CURE is a training-free concept unlearning framework that operates directly in the weight space of pre-trained diffusion models.<n>The Spectral Eraser identifies and isolates features unique to the undesired concept while preserving safe attributes.<n>CURE achieves a more efficient and thorough removal for targeted artistic styles, objects, identities, or explicit content.
  arXiv  Detail & Related papers  (2025-05-19T03:53:06Z)
• SPEED: Scalable, Precise, and Efficient Concept Erasure for Diffusion Models [56.83154571623655]
  We introduce SPEED, an efficient concept erasure approach that directly edits model parameters.<n>Speedy searches for a null space, a model editing space where parameter updates do not affect non-target concepts.<n>We successfully erase 100 concepts within only 5 seconds.
  arXiv  Detail & Related papers  (2025-03-10T14:40:01Z)
• Reliable and Efficient Concept Erasure of Text-to-Image Diffusion Models [76.39651111467832]
  We introduce Reliable and Efficient Concept Erasure (RECE), a novel approach that modifies the model in 3 seconds without necessitating additional fine-tuning. To mitigate inappropriate content potentially represented by derived embeddings, RECE aligns them with harmless concepts in cross-attention layers. The derivation and erasure of new representation embeddings are conducted iteratively to achieve a thorough erasure of inappropriate concepts.
  arXiv  Detail & Related papers  (2024-07-17T08:04:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.