High-Fidelity Diffusion Face Swapping with ID-Constrained Facial Conditioning

• URL: http://arxiv.org/abs/2503.22179v1
• Date: Fri, 28 Mar 2025 06:50:17 GMT
• Title: High-Fidelity Diffusion Face Swapping with ID-Constrained Facial Conditioning
• Authors: Dailan He, Xiahong Wang, Shulun Wang, Guanglu Song, Bingqi Ma, Hao Shao, Yu Liu, Hongsheng Li,
• Abstract summary: Face swapping aims to seamlessly transfer a source facial identity onto a target while preserving target attributes such as pose and expression.<n> Diffusion models, known for their superior generative capabilities, have recently shown promise in advancing face-swapping quality.<n>This paper addresses two key challenges in diffusion-based face swapping: the prioritized preservation of identity over target attributes and the inherent conflict between identity and attribute conditioning.
• Score: 39.09330483562798
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Face swapping aims to seamlessly transfer a source facial identity onto a target while preserving target attributes such as pose and expression. Diffusion models, known for their superior generative capabilities, have recently shown promise in advancing face-swapping quality. This paper addresses two key challenges in diffusion-based face swapping: the prioritized preservation of identity over target attributes and the inherent conflict between identity and attribute conditioning. To tackle these issues, we introduce an identity-constrained attribute-tuning framework for face swapping that first ensures identity preservation and then fine-tunes for attribute alignment, achieved through a decoupled condition injection. We further enhance fidelity by incorporating identity and adversarial losses in a post-training refinement stage. Our proposed identity-constrained diffusion-based face-swapping model outperforms existing methods in both qualitative and quantitative evaluations, demonstrating superior identity similarity and attribute consistency, achieving a new state-of-the-art performance in high-fidelity face swapping.

Related papers

• Diffusion-based Adversarial Identity Manipulation for Facial Privacy Protection [14.797807196805607]
  Face recognition has led to serious privacy concerns due to potential unauthorized surveillance and user tracking on social networks. Existing methods for enhancing privacy fail to generate natural face images that can protect facial privacy. We propose DiffAIM to generate natural and highly transferable adversarial faces against malicious FR systems.
  arXiv  Detail & Related papers  (2025-04-30T13:49:59Z)
• iFADIT: Invertible Face Anonymization via Disentangled Identity Transform [51.123936665445356]
  Face anonymization aims to conceal the visual identity of a face to safeguard the individual's privacy. This paper proposes a novel framework named iFADIT, an acronym for Invertible Face Anonymization via Disentangled Identity Transform.
  arXiv  Detail & Related papers  (2025-01-08T10:08:09Z)
• HiFiVFS: High Fidelity Video Face Swapping [35.49571526968986]
  Face swapping aims to generate results that combine the identity from the source with attributes from the target.<n>We propose a high fidelity video face swapping framework, which leverages the strong generative capability and temporal prior of Stable Video Diffusion.<n>Our method achieves state-of-the-art (SOTA) in video face swapping, both qualitatively and quantitatively.
  arXiv  Detail & Related papers  (2024-11-27T12:30:24Z)
• ID$^3$: Identity-Preserving-yet-Diversified Diffusion Models for Synthetic Face Recognition [60.15830516741776]
  Synthetic face recognition (SFR) aims to generate datasets that mimic the distribution of real face data. We introduce a diffusion-fueled SFR model termed $textID3$. $textID3$ employs an ID-preserving loss to generate diverse yet identity-consistent facial appearances.
  arXiv  Detail & Related papers  (2024-09-26T06:46:40Z)
• Disentangle Before Anonymize: A Two-stage Framework for Attribute-preserved and Occlusion-robust De-identification [55.741525129613535]
  "Disentangle Before Anonymize" is a novel two-stage Framework(DBAF) This framework includes a Contrastive Identity Disentanglement (CID) module and a Key-authorized Reversible Identity Anonymization (KRIA) module. Extensive experiments demonstrate that our method outperforms state-of-the-art de-identification approaches.
  arXiv  Detail & Related papers  (2023-11-15T08:59:02Z)
• FaceDancer: Pose- and Occlusion-Aware High Fidelity Face Swapping [62.38898610210771]
  We present a new single-stage method for subject face swapping and identity transfer, named FaceDancer. We have two major contributions: Adaptive Feature Fusion Attention (AFFA) and Interpreted Feature Similarity Regularization (IFSR)
  arXiv  Detail & Related papers  (2022-10-19T11:31:38Z)
• FICGAN: Facial Identity Controllable GAN for De-identification [34.38379234653657]
  We present Facial Identity Controllable GAN (FICGAN) for generating high-quality de-identified face images with ensured privacy protection. Based on the analysis, we develop FICGAN, an autoencoder-based conditional generative model that learns to disentangle the identity attributes from non-identity attributes on a face image.
  arXiv  Detail & Related papers  (2021-10-02T07:09:27Z)
• Dual-Attention GAN for Large-Pose Face Frontalization [59.689836951934694]
  We present a novel Dual-Attention Generative Adversarial Network (DA-GAN) for photo-realistic face frontalization. Specifically, a self-attention-based generator is introduced to integrate local features with their long-range dependencies. A novel face-attention-based discriminator is applied to emphasize local features of face regions.
  arXiv  Detail & Related papers  (2020-02-17T20:00:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.