Safeguarding Facial Identity against Diffusion-based Face Swapping via Cascading Pathway Disruption

• URL: http://arxiv.org/abs/2601.14738v1
• Date: Wed, 21 Jan 2026 07:52:56 GMT
• Title: Safeguarding Facial Identity against Diffusion-based Face Swapping via Cascading Pathway Disruption
• Authors: Liqin Wang, Qianyue Hu, Wei Lu, Xiangyang Luo,
• Abstract summary: We propose VoidFace, a systemic defense method that views face swapping as a coupled identity pathway.<n>We first introduce localization disruption and identity erasure to degrade physical regression and semantic embeddings, thereby impairing the accurate modeling of the source face.<n>We then intervene in the generative domain by decoupling attention mechanisms to sever identity injection, and corrupting intermediate diffusion features to prevent the reconstruction of source identity.
• Score: 21.37567715195999
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The rapid evolution of diffusion models has democratized face swapping but also raises concerns about privacy and identity security. Existing proactive defenses, often adapted from image editing attacks, prove ineffective in this context. We attribute this failure to an oversight of the structural resilience and the unique static conditional guidance mechanism inherent in face swapping systems. To address this, we propose VoidFace, a systemic defense method that views face swapping as a coupled identity pathway. By injecting perturbations at critical bottlenecks, VoidFace induces cascading disruption throughout the pipeline. Specifically, we first introduce localization disruption and identity erasure to degrade physical regression and semantic embeddings, thereby impairing the accurate modeling of the source face. We then intervene in the generative domain by decoupling attention mechanisms to sever identity injection, and corrupting intermediate diffusion features to prevent the reconstruction of source identity. To ensure visual imperceptibility, we perform adversarial search in the latent manifold, guided by a perceptual adaptive strategy to balance attack potency with image quality. Extensive experiments show that VoidFace outperforms existing defenses across various diffusion-based swapping models, while producing adversarial faces with superior visual quality.

Related papers

• SIDeR: Semantic Identity Decoupling for Unrestricted Face Privacy [53.75084833636302]
  We propose SIDeR, a Semantic decoupling-driven framework for unrestricted face privacy protection.<n> SIDeR decomposes a facial image into a machine-recognizable identity feature vector and a visually perceptible semantic appearance component.<n>For authorized access, SIDeR can be restored to its original form when the correct password is provided.
  arXiv  Detail & Related papers  (2026-02-04T19:30:48Z)
• Beauty and the Beast: Imperceptible Perturbations Against Diffusion-Based Face Swapping via Directional Attribute Editing [21.375408098632615]
  Diffusion-based face swapping achieves state-of-the-art performance, yet it exacerbates the potential harm of malicious face swapping to violate portraiture right or undermine personal reputation.<n>We propose FaceDefense, an enhanced proactive defense framework against diffusion-based face swapping.<n>Our method introduces a new diffusion loss to strengthen the defensive efficacy of adversarial examples, and employs a directional facial attribute editing to restore perturbation-induced distortions.
  arXiv  Detail & Related papers  (2026-01-30T09:24:47Z)
• Diffusion-based Adversarial Identity Manipulation for Facial Privacy Protection [14.797807196805607]
  Face recognition has led to serious privacy concerns due to potential unauthorized surveillance and user tracking on social networks.<n>Existing methods for enhancing privacy fail to generate natural face images that can protect facial privacy.<n>We propose DiffAIM to generate natural and highly transferable adversarial faces against malicious FR systems.
  arXiv  Detail & Related papers  (2025-04-30T13:49:59Z)
• High-Fidelity Diffusion Face Swapping with ID-Constrained Facial Conditioning [39.09330483562798]
  Face swapping aims to seamlessly transfer a source facial identity onto a target while preserving target attributes such as pose and expression.<n> Diffusion models, known for their superior generative capabilities, have recently shown promise in advancing face-swapping quality.<n>This paper addresses two key challenges in diffusion-based face swapping: the prioritized preservation of identity over target attributes and the inherent conflict between identity and attribute conditioning.
  arXiv  Detail & Related papers  (2025-03-28T06:50:17Z)
• NullSwap: Proactive Identity Cloaking Against Deepfake Face Swapping [8.284351945561099]
  We analyze the essence of Deepfake face swapping and argue the necessity of protecting source identities rather than target images.<n>We propose NullSwap, a novel proactive defense approach that cloaks source image identities and nullifies face swapping under a pure black-box scenario.<n> Experiments demonstrate the outstanding ability of our approach to fool various identity recognition models.
  arXiv  Detail & Related papers  (2025-03-24T13:49:39Z)
• Enhancing Facial Privacy Protection via Weakening Diffusion Purification [36.33027625681024]
  Social media has led to the widespread sharing of individual portrait images, which pose serious privacy risks.<n>Recent methods employ diffusion models to generate adversarial face images for privacy protection.<n>We propose learning unconditional embeddings to increase the learning capacity for adversarial modifications.<n>We integrate an identity-preserving structure to maintain structural consistency between the original and generated images.
  arXiv  Detail & Related papers  (2025-03-13T13:27:53Z)
• iFADIT: Invertible Face Anonymization via Disentangled Identity Transform [51.123936665445356]
  Face anonymization aims to conceal the visual identity of a face to safeguard the individual's privacy.<n>This paper proposes a novel framework named iFADIT, an acronym for Invertible Face Anonymization via Disentangled Identity Transform.
  arXiv  Detail & Related papers  (2025-01-08T10:08:09Z)
• OSDFace: One-Step Diffusion Model for Face Restoration [72.5045389847792]
  Diffusion models have demonstrated impressive performance in face restoration.<n>We propose OSDFace, a novel one-step diffusion model for face restoration.<n>Results demonstrate that OSDFace surpasses current state-of-the-art (SOTA) methods in both visual quality and quantitative metrics.
  arXiv  Detail & Related papers  (2024-11-26T07:07:48Z)
• DiffusionFake: Enhancing Generalization in Deepfake Detection via Guided Stable Diffusion [94.46904504076124]
  Deepfake technology has made face swapping highly realistic, raising concerns about the malicious use of fabricated facial content. Existing methods often struggle to generalize to unseen domains due to the diverse nature of facial manipulations. We introduce DiffusionFake, a novel framework that reverses the generative process of face forgeries to enhance the generalization of detection models.
  arXiv  Detail & Related papers  (2024-10-06T06:22:43Z)
• ID-Guard: A Universal Framework for Combating Facial Manipulation via Breaking Identification [60.73617868629575]
  misuse of deep learning-based facial manipulation poses a significant threat to civil rights.<n>To prevent this fraud at its source, proactive defense has been proposed to disrupt the manipulation process.<n>This paper proposes a universal framework for combating facial manipulation, termed ID-Guard.
  arXiv  Detail & Related papers  (2024-09-20T09:30:08Z)
• Adv-Diffusion: Imperceptible Adversarial Face Identity Attack via Latent Diffusion Model [61.53213964333474]
  We propose a unified framework Adv-Diffusion that can generate imperceptible adversarial identity perturbations in the latent space but not the raw pixel space. Specifically, we propose the identity-sensitive conditioned diffusion generative model to generate semantic perturbations in the surroundings. The designed adaptive strength-based adversarial perturbation algorithm can ensure both attack transferability and stealthiness.
  arXiv  Detail & Related papers  (2023-12-18T15:25:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.