Diffusion-based Adversarial Identity Manipulation for Facial Privacy Protection

• URL: http://arxiv.org/abs/2504.21646v1
• Date: Wed, 30 Apr 2025 13:49:59 GMT
• Title: Diffusion-based Adversarial Identity Manipulation for Facial Privacy Protection
• Authors: Liqin Wang, Qianyue Hu, Wei Lu, Xiangyang Luo,
• Abstract summary: Face recognition has led to serious privacy concerns due to potential unauthorized surveillance and user tracking on social networks.<n>Existing methods for enhancing privacy fail to generate natural face images that can protect facial privacy.<n>We propose DiffAIM to generate natural and highly transferable adversarial faces against malicious FR systems.
• Score: 14.797807196805607
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The success of face recognition (FR) systems has led to serious privacy concerns due to potential unauthorized surveillance and user tracking on social networks. Existing methods for enhancing privacy fail to generate natural face images that can protect facial privacy. In this paper, we propose diffusion-based adversarial identity manipulation (DiffAIM) to generate natural and highly transferable adversarial faces against malicious FR systems. To be specific, we manipulate facial identity within the low-dimensional latent space of a diffusion model. This involves iteratively injecting gradient-based adversarial identity guidance during the reverse diffusion process, progressively steering the generation toward the desired adversarial faces. The guidance is optimized for identity convergence towards a target while promoting semantic divergence from the source, facilitating effective impersonation while maintaining visual naturalness. We further incorporate structure-preserving regularization to preserve facial structure consistency during manipulation. Extensive experiments on both face verification and identification tasks demonstrate that compared with the state-of-the-art, DiffAIM achieves stronger black-box attack transferability while maintaining superior visual quality. We also demonstrate the effectiveness of the proposed approach for commercial FR APIs, including Face++ and Aliyun.

Related papers

• Enhancing Facial Privacy Protection via Weakening Diffusion Purification [36.33027625681024]
  Social media has led to the widespread sharing of individual portrait images, which pose serious privacy risks.<n>Recent methods employ diffusion models to generate adversarial face images for privacy protection.<n>We propose learning unconditional embeddings to increase the learning capacity for adversarial modifications.<n>We integrate an identity-preserving structure to maintain structural consistency between the original and generated images.
  arXiv  Detail & Related papers  (2025-03-13T13:27:53Z)
• iFADIT: Invertible Face Anonymization via Disentangled Identity Transform [51.123936665445356]
  Face anonymization aims to conceal the visual identity of a face to safeguard the individual's privacy.<n>This paper proposes a novel framework named iFADIT, an acronym for Invertible Face Anonymization via Disentangled Identity Transform.
  arXiv  Detail & Related papers  (2025-01-08T10:08:09Z)
• ID-Guard: A Universal Framework for Combating Facial Manipulation via Breaking Identification [60.73617868629575]
  misuse of deep learning-based facial manipulation poses a significant threat to civil rights.<n>To prevent this fraud at its source, proactive defense has been proposed to disrupt the manipulation process.<n>This paper proposes a universal framework for combating facial manipulation, termed ID-Guard.
  arXiv  Detail & Related papers  (2024-09-20T09:30:08Z)
• Transferable Adversarial Facial Images for Privacy Protection [15.211743719312613]
  We present a novel face privacy protection scheme with improved transferability while maintain high visual quality. We first exploit global adversarial latent search to traverse the latent space of the generative model. We then introduce a key landmark regularization module to preserve the visual identity information.
  arXiv  Detail & Related papers  (2024-07-18T02:16:11Z)
• Diff-Privacy: Diffusion-based Face Privacy Protection [58.1021066224765]
  In this paper, we propose a novel face privacy protection method based on diffusion models, dubbed Diff-Privacy. Specifically, we train our proposed multi-scale image inversion module (MSI) to obtain a set of SDM format conditional embeddings of the original image. Based on the conditional embeddings, we design corresponding embedding scheduling strategies and construct different energy functions during the denoising process to achieve anonymization and visual identity information hiding.
  arXiv  Detail & Related papers  (2023-09-11T09:26:07Z)
• DiffProtect: Generate Adversarial Examples with Diffusion Models for Facial Privacy Protection [64.77548539959501]
  DiffProtect produces more natural-looking encrypted images than state-of-the-art methods. It achieves significantly higher attack success rates, e.g., 24.5% and 25.1% absolute improvements on the CelebA-HQ and FFHQ datasets.
  arXiv  Detail & Related papers  (2023-05-23T02:45:49Z)
• Attribute-Guided Encryption with Facial Texture Masking [64.77548539959501]
  We propose Attribute Guided Encryption with Facial Texture Masking to protect users from unauthorized facial recognition systems. Our proposed method produces more natural-looking encrypted images than state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-05-22T23:50:43Z)
• DuetFace: Collaborative Privacy-Preserving Face Recognition via Channel Splitting in the Frequency Domain [23.4606547767188]
  DuetFace is a privacy-preserving face recognition method that employs collaborative inference in the frequency domain. The proposed method achieves a comparable recognition accuracy and cost to the unprotected ArcFace and outperforms the state-of-the-art privacy-preserving methods.
  arXiv  Detail & Related papers  (2022-07-15T08:35:44Z)
• Protecting Facial Privacy: Generating Adversarial Identity Masks via Style-robust Makeup Transfer [24.25863892897547]
  adversarial makeup transfer GAN (AMT-GAN) is a novel face protection method aiming at constructing adversarial face images. In this paper, we introduce a new regularization module along with a joint training strategy to reconcile the conflicts between the adversarial noises and the cycle consistence loss in makeup transfer.
  arXiv  Detail & Related papers  (2022-03-07T03:56:17Z)
• Towards Face Encryption by Generating Adversarial Identity Masks [53.82211571716117]
  We propose a targeted identity-protection iterative method (TIP-IM) to generate adversarial identity masks. TIP-IM provides 95%+ protection success rate against various state-of-the-art face recognition models.
  arXiv  Detail & Related papers  (2020-03-15T12:45:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.