PIU: Proximity-guided Identity Unlearning in ID-Conditioned Diffusion Models

• URL: http://arxiv.org/abs/2605.22311v1
• Date: Thu, 21 May 2026 10:55:56 GMT
• Title: PIU: Proximity-guided Identity Unlearning in ID-Conditioned Diffusion Models
• Abstract summary: We study identity unlearning in Arc2Face, a state-of-the-art identity-conditioned latent diffusion model for face generation.<n>We introduce Proximity-guided Identity Unlearning (PIU), an anchor-guided framework for identity unlearning.
• Score: 2.260041671242786
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Identity-conditioned diffusion models enable high-quality and identity-consistent face generation, but they also raise severe privacy concerns, as models may continue to synthesize individuals despite their right to be forgotten. While machine unlearning has been extensively studied for concept and data removal, identity unlearning remains largely unexplored, particularly in models conditioned directly on identity embeddings rather than text prompts. In this work, we study identity unlearning in Arc2Face, a state-of-the-art identity-conditioned latent diffusion model for face generation, and introduce Proximity-guided Identity Unlearning (PIU), an anchor-guided framework for identity unlearning. Specifically, we formulate identity removal as an identity replacement objective that reassigns the source identity to a selected anchor identity in the learned identity space, and we complement it with a proximity-based anchor selection strategy motivated by the geometry of ArcFace representations. We further show that effective unlearning can be achieved through localized fine-tuning of a small subset of identity-sensitive cross-attention layers. Experiments across many target identities show that our framework effectively suppresses generation of the target identity while preserving realism and identity consistency for retained identities, as validated by improved performance on unlearning and image-quality metrics, together with qualitative evaluation. The source code for the PIU framework is publicly available at https://github.com/edgarcancinoe/piu_unlearning .

Related papers

• Diff-ID: Identity Consistent Facial Image Generation and Morphing via Diffusion Models [42.651660700496606]
  We introduce Diff-ID, a diffusion based framework that enforces identity consistency while delivering photorealistic quality.<n>We propose a pseudo discriminator loss based on ArcFace cosine similarity with exponential timestep weighting to reinforce identity perceptual fidelity.<n>Experiments show Diff-ID does not exceed InstantID in raw ArcFace Face Similarity, but achieves substantially lower FID and the strongest FIQ based identity--realism trade off among the evaluated methods.
  arXiv  Detail & Related papers  (2026-07-27T21:13:29Z)
• Face inpainting with Identity Preserving Latent Diffusion Models [10.348748950921667]
  ID-ControlNet is an identity-preserving face inpainting framework built upon latent diffusion models.<n>We introduce an identity consistency and triplet loss training strategy that explicitly enforces alignment between the generated face and the target identity representation.
  arXiv  Detail & Related papers  (2026-05-15T23:19:43Z)
• Identity-Decoupled Anonymization for Visual Evidence in Multi-modal Retrieval-Augmented Generation [9.39195684989942]
  Identity-Decoupled MRAG is a framework that interposes a generative anonymization module between retrieval and generation.<n>Our approach consists of three components: (i)a disentangled variational encoder that factorizes each face into an identity code and a spatially-structured attribute code, regularized by a mutual-information penalty and a gradient-based independence term; (ii)a manifold-aware rejection sampler that replaces the identity code with a synthetic one guaranteed to be both distinct from the original and realistic; and (iii)a conditional latent diffusion generator that synthesizes the anonymized face from the replacement identity and the
  arXiv  Detail & Related papers  (2026-04-26T07:42:33Z)
• Few-Shot Generative Model Adaption via Identity Injection and Preservation [28.832140076917515]
  Training generative models with limited data presents severe challenges of mode collapse.<n>A common approach is to adapt a large pretrained generative model upon a target domain with very few samples, known as few-shot generative model adaptation.<n>We propose Identity Injection and Preservation (I$2$P), which leverages identity injection and consistency alignment to preserve the source identity knowledge.
  arXiv  Detail & Related papers  (2026-03-24T08:59:29Z)
• Face Identity Unlearning for Retrieval via Embedding Dispersion [0.0]
  Face recognition systems rely on learning highly discriminative and compact identity clusters to enable accurate retrieval.<n>In this work, we study the problem of face identity unlearning for retrieval systems and present its inherent challenges.<n>The primary challenge is to achieve this forgetting effect while preserving the discriminative structure of the embedding space.
  arXiv  Detail & Related papers  (2025-12-15T13:35:28Z)
• Beyond Inference Intervention: Identity-Decoupled Diffusion for Face Anonymization [55.29071072675132]
  Face anonymization aims to conceal identity information while preserving non-identity attributes.<n>We propose textbfIDsuperscript2Face, a training-centric anonymization framework.<n>We show that IDtextsuperscript2Face outperforms existing methods in visual quality, identity suppression, and utility preservation.
  arXiv  Detail & Related papers  (2025-10-28T09:28:12Z)
• WithAnyone: Towards Controllable and ID Consistent Image Generation [83.55786496542062]
  Identity-consistent generation has become an important focus in text-to-image research.<n>We develop a large-scale paired dataset tailored for multi-person scenarios.<n>We propose a novel training paradigm with a contrastive identity loss that leverages paired data to balance fidelity with diversity.
  arXiv  Detail & Related papers  (2025-10-16T17:59:54Z)
• Foundation Cures Personalization: Improving Personalized Models' Prompt Consistency via Hidden Foundation Knowledge [49.36669870661573]
  We propose FreeCure, a framework that improves the prompt consistency of personalization models.<n>We introduce a novel foundation-aware self-attention module, coupled with an inversion-based process to bring well-aligned attribute information to the personalization process.
  arXiv  Detail & Related papers  (2024-11-22T15:21:38Z)
• ID-Aligner: Enhancing Identity-Preserving Text-to-Image Generation with Reward Feedback Learning [57.91881829308395]
  Identity-preserving text-to-image generation (ID-T2I) has received significant attention due to its wide range of application scenarios like AI portrait and advertising. We present textbfID-Aligner, a general feedback learning framework to enhance ID-T2I performance.
  arXiv  Detail & Related papers  (2024-04-23T18:41:56Z)
• Disentangle Before Anonymize: A Two-stage Framework for Attribute-preserved and Occlusion-robust De-identification [55.741525129613535]
  "Disentangle Before Anonymize" is a novel two-stage Framework(DBAF)<n>This framework includes a Contrastive Identity Disentanglement (CID) module and a Key-authorized Reversible Identity Anonymization (KRIA) module.<n>Extensive experiments demonstrate that our method outperforms state-of-the-art de-identification approaches.
  arXiv  Detail & Related papers  (2023-11-15T08:59:02Z)
• FaceDancer: Pose- and Occlusion-Aware High Fidelity Face Swapping [62.38898610210771]
  We present a new single-stage method for subject face swapping and identity transfer, named FaceDancer. We have two major contributions: Adaptive Feature Fusion Attention (AFFA) and Interpreted Feature Similarity Regularization (IFSR)
  arXiv  Detail & Related papers  (2022-10-19T11:31:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.