IREU: Identity-Related Encoder-Only Unlearning for Customized Portrait Generation

• URL: http://arxiv.org/abs/2606.29880v1
• Date: Mon, 29 Jun 2026 07:18:01 GMT
• Title: IREU: Identity-Related Encoder-Only Unlearning for Customized Portrait Generation
• Abstract summary: Customized Portrait Generation (CPG) technologies have been widely used to generate high-fidelity person images.<n>These methods pose significant privacy risks by spreading fake visual information.<n>In this work we investigate the identity unlearning problem for CPG.
• Score: 19.143405461663175
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Customized Portrait Generation (CPG) technologies have been widely used to generate high-fidelity person images given an input image indicating the identity and a text prompt indicating the required edits. Yet these methods pose significant privacy risks by spreading fake visual information. Against such risks, each public generator should be able to suppress its generation ability for a particular person when requested. Therefore, in this work we investigate the identity unlearning problem for CPG. Since there are no previous methods in this field, we propose a simple baseline that updates the image encoder by minimizing identity similarity between generated and input images for target identities to be unlearned, while maximizing it for identities to be retained. However, we find such a global perturbation in the feature space harms the fidelity of generated images for other identities to be retained. To solve this problem, we propose a novel method IREU, which first locates identity-related features in an offline manner and then only performs feature perturbations on them. The experimental results show that our proposed method IREU achieves better identity unlearning performance for target identities to be unlearned, and also keeps high fidelity for other identities to be retained. In addition, our unlearned image encoder is generalizable across different generators with the same encoder without fine-tuning, which is friendly for deployment in practice.

Related papers

• Diff-ID: Identity Consistent Facial Image Generation and Morphing via Diffusion Models [42.651660700496606]
  We introduce Diff-ID, a diffusion based framework that enforces identity consistency while delivering photorealistic quality.<n>We propose a pseudo discriminator loss based on ArcFace cosine similarity with exponential timestep weighting to reinforce identity perceptual fidelity.<n>Experiments show Diff-ID does not exceed InstantID in raw ArcFace Face Similarity, but achieves substantially lower FID and the strongest FIQ based identity--realism trade off among the evaluated methods.
  arXiv  Detail & Related papers  (2026-07-27T21:13:29Z)
• Identity-Decoupled Anonymization for Visual Evidence in Multi-modal Retrieval-Augmented Generation [9.39195684989942]
  Identity-Decoupled MRAG is a framework that interposes a generative anonymization module between retrieval and generation.<n>Our approach consists of three components: (i)a disentangled variational encoder that factorizes each face into an identity code and a spatially-structured attribute code, regularized by a mutual-information penalty and a gradient-based independence term; (ii)a manifold-aware rejection sampler that replaces the identity code with a synthetic one guaranteed to be both distinct from the original and realistic; and (iii)a conditional latent diffusion generator that synthesizes the anonymized face from the replacement identity and the
  arXiv  Detail & Related papers  (2026-04-26T07:42:33Z)
• Adapter Shield: A Unified Framework with Built-in Authentication for Preventing Unauthorized Zero-Shot Image-to-Image Generation [74.5813283875938]
  Zero-shot image-to-image generation poses substantial risks related to intellectual property violations.<n>This work presents Adapter Shield, the first universal and authentication-integrated solution aimed at defending personal images from misuse.<n>Our method surpasses existing state-of-the-art defenses in blocking unauthorized zero-shot image synthesis.
  arXiv  Detail & Related papers  (2025-11-25T04:49:16Z)
• Beyond Inference Intervention: Identity-Decoupled Diffusion for Face Anonymization [55.29071072675132]
  Face anonymization aims to conceal identity information while preserving non-identity attributes.<n>We propose textbfIDsuperscript2Face, a training-centric anonymization framework.<n>We show that IDtextsuperscript2Face outperforms existing methods in visual quality, identity suppression, and utility preservation.
  arXiv  Detail & Related papers  (2025-10-28T09:28:12Z)
• IDProtector: An Adversarial Noise Encoder to Protect Against ID-Preserving Image Generation [21.06493827123594]
  We introduce IDProtector, an adversarial noise encoder that applies imperceptible adversarial noise to portrait photos in a single forward pass.<n>Our approach offers universal protection for portraits against multiple state-of-the-art encoder-based methods, including InstantID, IP-Adapter, and PhotoMaker.
  arXiv  Detail & Related papers  (2024-12-16T10:27:48Z)
• Generative Unlearning for Any Identity [6.872154067622779]
  In certain domains related to privacy issues, advanced generative models along with strong inversion methods can lead to potential misuses. We propose an essential yet under-explored task called generative identity unlearning, which steers the model not to generate an image of a specific identity. We propose a novel framework, Generative Unlearning for Any Identity (GUIDE), which prevents the reconstruction of a specific identity by unlearning the generator with only a single image.
  arXiv  Detail & Related papers  (2024-05-16T08:00:55Z)
• ID-Aligner: Enhancing Identity-Preserving Text-to-Image Generation with Reward Feedback Learning [57.91881829308395]
  Identity-preserving text-to-image generation (ID-T2I) has received significant attention due to its wide range of application scenarios like AI portrait and advertising. We present textbfID-Aligner, a general feedback learning framework to enhance ID-T2I performance.
  arXiv  Detail & Related papers  (2024-04-23T18:41:56Z)
• StableIdentity: Inserting Anybody into Anywhere at First Sight [57.99693188913382]
  We propose StableIdentity, which allows identity-consistent recontextualization with just one face image. We are the first to directly inject the identity learned from a single image into video/3D generation without finetuning.
  arXiv  Detail & Related papers  (2024-01-29T09:06:15Z)
• Disentangle Before Anonymize: A Two-stage Framework for Attribute-preserved and Occlusion-robust De-identification [55.741525129613535]
  "Disentangle Before Anonymize" is a novel two-stage Framework(DBAF)<n>This framework includes a Contrastive Identity Disentanglement (CID) module and a Key-authorized Reversible Identity Anonymization (KRIA) module.<n>Extensive experiments demonstrate that our method outperforms state-of-the-art de-identification approaches.
  arXiv  Detail & Related papers  (2023-11-15T08:59:02Z)
• Identity Encoder for Personalized Diffusion [57.1198884486401]
  We propose an encoder-based approach for personalization. We learn an identity encoder which can extract an identity representation from a set of reference images of a subject. We show that our approach consistently outperforms existing fine-tuning based approach in both image generation and reconstruction.
  arXiv  Detail & Related papers  (2023-04-14T23:32:24Z)
• Disguise without Disruption: Utility-Preserving Face De-Identification [40.484745636190034]
  We introduce Disguise, a novel algorithm that seamlessly de-identifies facial images while ensuring the usability of the modified data. Our method involves extracting and substituting depicted identities with synthetic ones, generated using variational mechanisms to maximize obfuscation and non-invertibility. We extensively evaluate our method using multiple datasets, demonstrating a higher de-identification rate and superior consistency compared to prior approaches in various downstream tasks.
  arXiv  Detail & Related papers  (2023-03-23T13:50:46Z)
• Attribute-preserving Face Dataset Anonymization via Latent Code Optimization [64.4569739006591]
  We present a task-agnostic anonymization procedure that directly optimize the images' latent representation in the latent space of a pre-trained GAN. We demonstrate through a series of experiments that our method is capable of anonymizing the identity of the images whilst -- crucially -- better-preserving the facial attributes.
  arXiv  Detail & Related papers  (2023-03-20T17:34:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.