Diff-PC: Identity-preserving and 3D-aware Controllable Diffusion for Zero-shot Portrait Customization

• URL: http://arxiv.org/abs/2602.00639v1
• Date: Sat, 31 Jan 2026 10:15:41 GMT
• Title: Diff-PC: Identity-preserving and 3D-aware Controllable Diffusion for Zero-shot Portrait Customization
• Authors: Yifang Xu, Benxiang Zhai, Chenyu Zhang, Ming Li, Yang Li, Sidan Du,
• Abstract summary: Diff-PC is a diffusion-based framework for zero-shot portrait customization (PC)<n>It generates realistic portraits with high ID fidelity, specified facial attributes, and diverse backgrounds.<n>Our approach employs the 3D face predictor to reconstruct the 3D-aware facial priors.
• Score: 13.128154695283477
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Portrait customization (PC) has recently garnered significant attention due to its potential applications. However, existing PC methods lack precise identity (ID) preservation and face control. To address these tissues, we propose Diff-PC, a diffusion-based framework for zero-shot PC, which generates realistic portraits with high ID fidelity, specified facial attributes, and diverse backgrounds. Specifically, our approach employs the 3D face predictor to reconstruct the 3D-aware facial priors encompassing the reference ID, target expressions, and poses. To capture fine-grained face details, we design ID-Encoder that fuses local and global facial features. Subsequently, we devise ID-Ctrl using the 3D face to guide the alignment of ID features. We further introduce ID-Injector to enhance ID fidelity and facial controllability. Finally, training on our collected ID-centric dataset improves face similarity and text-to-image (T2I) alignment. Extensive experiments demonstrate that Diff-PC surpasses state-of-the-art methods in ID preservation, facial control, and T2I consistency. Furthermore, our method is compatible with multi-style foundation models.

Related papers

• Optimizing ID Consistency in Multimodal Large Models: Facial Restoration via Alignment, Entanglement, and Disentanglement [54.199726425201895]
  Multimodal editing large models have demonstrated powerful editing capabilities across diverse tasks.<n>Current facial ID preservation methods struggle to achieve consistent restoration of both facial identity and edited element IP.<n>We propose EditedID, an Alignment-Disentanglement-Entanglement framework for robust identity-specific facial restoration.
  arXiv  Detail & Related papers  (2026-02-21T08:24:42Z)
• HiFi-Portrait: Zero-shot Identity-preserved Portrait Generation with High-fidelity Multi-face Fusion [12.382436378979564]
  HiFi-Portrait is a high-fidelity method for zero-shot portrait generation.<n>Our method surpasses the SOTA approaches in face similarity and controllability.
  arXiv  Detail & Related papers  (2025-12-16T16:17:46Z)
• DiffSwap++: 3D Latent-Controlled Diffusion for Identity-Preserving Face Swapping [16.846179110602737]
  We propose DiffSwap++, a novel diffusion-based face-swapping pipeline that incorporates 3D facial latent features during training.<n>Our method enhances geometric consistency and improves the disentanglement of facial identity from appearance attributes.<n>Experiments on CelebA, FFHQ, and CelebV-Text demonstrate that DiffSwap++ outperforms prior methods in preserving source identity while maintaining target pose and expression.
  arXiv  Detail & Related papers  (2025-11-04T18:56:49Z)
• Personalized Face Super-Resolution with Identity Decoupling and Fitting [50.473357681579664]
  In extreme degradation scenarios, critical attributes and ID information are often severely lost in the input image.<n>Existing methods tend to generate hallucinated faces under such conditions, producing restored images lacking authentic ID constraints.<n>We propose a novel FSR method with Identity Decoupling and Fitting (IDFSR) to enhance ID restoration under large scaling factors.
  arXiv  Detail & Related papers  (2025-08-13T02:33:11Z)
• G2Face: High-Fidelity Reversible Face Anonymization via Generative and Geometric Priors [71.69161292330504]
  Reversible face anonymization seeks to replace sensitive identity information in facial images with synthesized alternatives. This paper introduces Gtextsuperscript2Face, which leverages both generative and geometric priors to enhance identity manipulation. Our method outperforms existing state-of-the-art techniques in face anonymization and recovery, while preserving high data utility.
  arXiv  Detail & Related papers  (2024-08-18T12:36:47Z)
• ID-to-3D: Expressive ID-guided 3D Heads via Score Distillation Sampling [96.87575334960258]
  ID-to-3D is a method to generate identity- and text-guided 3D human heads with disentangled expressions. Results achieve an unprecedented level of identity-consistent and high-quality texture and geometry generation.
  arXiv  Detail & Related papers  (2024-05-26T13:36:45Z)
• ConsistentID: Portrait Generation with Multimodal Fine-Grained Identity Preserving [64.90148669690228]
  ConsistentID is an innovative method crafted for diverseidentity-preserving portrait generation under fine-grained multimodal facial prompts.<n>We present a fine-grained portrait dataset, FGID, with over 500,000 facial images, offering greater diversity and comprehensiveness than existing public facial datasets.
  arXiv  Detail & Related papers  (2024-04-25T17:23:43Z)
• StableIdentity: Inserting Anybody into Anywhere at First Sight [57.99693188913382]
  We propose StableIdentity, which allows identity-consistent recontextualization with just one face image. We are the first to directly inject the identity learned from a single image into video/3D generation without finetuning.
  arXiv  Detail & Related papers  (2024-01-29T09:06:15Z)
• A Systematical Solution for Face De-identification [6.244117712209321]
  In different tasks, people have various requirements for face de-identification (De-ID) We propose a systematical solution compatible for these De-ID operations. Our method can flexibly de-identify the face data in various ways and the processed images have high image quality.
  arXiv  Detail & Related papers  (2021-07-19T02:02:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.