Adv-TGD: Adversarial Text-Guided Diffusion for Face Recognition Impersonation Attacks

• URL: http://arxiv.org/abs/2606.11615v2
• Date: Tue, 16 Jun 2026 23:04:52 GMT
• Title: Adv-TGD: Adversarial Text-Guided Diffusion for Face Recognition Impersonation Attacks
• Abstract summary: We propose Adv-TGD, a generative adversarial attack framework.<n>It synthesizes photorealistic faces capable of impersonating target identities and deceiving face recognition systems.<n> Adv-TGD attains an average attack success rate (ASR) of 85.90% across IR152, IRSE50, MobileFace, and FaceNet.
• Score: 2.1485350418225244
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The widespread adoption of face recognition (FR) technologies raises serious privacy concerns, as facial data can be exploited without consent. To address this challenge, we propose Adv-TGD, a generative adversarial attack framework that synthesizes photorealistic faces capable of impersonating target identities and deceiving face recognition systems. Built upon Stable Diffusion v2.1, Adv-TGD performs per-sample LoRA fine-tuning conditioned on concise textual prompts to generate natural yet adversarially manipulated identities. Unlike conventional identity attack approaches, our method optimizes lightweight cross-attention adapters for each source-target pair within a fixed-timestep denoising process. Latent blending is constrained by a face-local heatmap mask to ensure spatially precise identity manipulation while preserving non-sensitive regions. We introduce a composite objective that integrates masked epsilon-MSE reconstruction, thresholded identity divergence in FR embedding space, directional feature alignment, and source-similarity suppression to balance adversarial attack and visual realism. Optionally, LLaVA-generated attribute prompts enhance fine-grained semantic details without reintroducing identity cues. Under the black-box evaluation protocol, Adv-TGD attains an average attack success rate (ASR) of 85.90% across IR152, IRSE50, MobileFace, and FaceNet, surpassing the semantic SOTA baseline Adv-CPG by 6.25 points, the diffusion-based makeup method DiffAIM by 3 points, and the noise-based P3-Mask by 16 points. Despite its strong attack efficacy, Adv-TGD preserves high visual fidelity (PSNR = 28.18 dB, SSIM = 0.981). Furthermore, we demonstrate the flexibility of our framework by successfully extending it to in-the-wild datasets (LADN), general object classification (ImageNet), and transformer-based diffusion models (FLUX.1).

Related papers

• DiffAttack: Evasion Attacks Against Face Recognition via Latent Diffusion Models [2.1485350418225244]
  We present a novel approach for adversarial face generation via latent-space optimization.<n>We leverage latent diffusion models directly to guide generation toward target identity embeddings, as measured by a face recognition model.<n>Our proposed framework has been evaluated on standard benchmarks, such as the FFHQ and CelebA-HQ datasets.
  arXiv  Detail & Related papers  (2026-07-31T01:33:53Z)
• Face inpainting with Identity Preserving Latent Diffusion Models [10.348748950921667]
  ID-ControlNet is an identity-preserving face inpainting framework built upon latent diffusion models.<n>We introduce an identity consistency and triplet loss training strategy that explicitly enforces alignment between the generated face and the target identity representation.
  arXiv  Detail & Related papers  (2026-05-15T23:19:43Z)
• SIDeR: Semantic Identity Decoupling for Unrestricted Face Privacy [53.75084833636302]
  We propose SIDeR, a Semantic decoupling-driven framework for unrestricted face privacy protection.<n> SIDeR decomposes a facial image into a machine-recognizable identity feature vector and a visually perceptible semantic appearance component.<n>For authorized access, SIDeR can be restored to its original form when the correct password is provided.
  arXiv  Detail & Related papers  (2026-02-04T19:30:48Z)
• A Dual-Branch CNN for Robust Detection of AI-Generated Facial Forgeries [4.313893060699182]
  Face forgery techniques pose significant threats to AI security, digital media integrity, and public trust.<n>We propose a novel dual-branch convolutional neural network for face forgery detection.<n>We evaluate our model on the DiFF benchmark, which includes forged images generated from four representative methods.
  arXiv  Detail & Related papers  (2025-10-28T17:06:40Z)
• Diffusion-based Adversarial Identity Manipulation for Facial Privacy Protection [14.797807196805607]
  Face recognition has led to serious privacy concerns due to potential unauthorized surveillance and user tracking on social networks.<n>Existing methods for enhancing privacy fail to generate natural face images that can protect facial privacy.<n>We propose DiffAIM to generate natural and highly transferable adversarial faces against malicious FR systems.
  arXiv  Detail & Related papers  (2025-04-30T13:49:59Z)
• ID$^3$: Identity-Preserving-yet-Diversified Diffusion Models for Synthetic Face Recognition [60.15830516741776]
  Synthetic face recognition (SFR) aims to generate datasets that mimic the distribution of real face data. We introduce a diffusion-fueled SFR model termed $textID3$. $textID3$ employs an ID-preserving loss to generate diverse yet identity-consistent facial appearances.
  arXiv  Detail & Related papers  (2024-09-26T06:46:40Z)
• Imperceptible Face Forgery Attack via Adversarial Semantic Mask [59.23247545399068]
  We propose an Adversarial Semantic Mask Attack framework (ASMA) which can generate adversarial examples with good transferability and invisibility. Specifically, we propose a novel adversarial semantic mask generative model, which can constrain generated perturbations in local semantic regions for good stealthiness.
  arXiv  Detail & Related papers  (2024-06-16T10:38:11Z)
• Adv-Diffusion: Imperceptible Adversarial Face Identity Attack via Latent Diffusion Model [61.53213964333474]
  We propose a unified framework Adv-Diffusion that can generate imperceptible adversarial identity perturbations in the latent space but not the raw pixel space. Specifically, we propose the identity-sensitive conditioned diffusion generative model to generate semantic perturbations in the surroundings. The designed adaptive strength-based adversarial perturbation algorithm can ensure both attack transferability and stealthiness.
  arXiv  Detail & Related papers  (2023-12-18T15:25:23Z)
• Attribute-Guided Encryption with Facial Texture Masking [64.77548539959501]
  We propose Attribute Guided Encryption with Facial Texture Masking to protect users from unauthorized facial recognition systems. Our proposed method produces more natural-looking encrypted images than state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-05-22T23:50:43Z)
• Dual Spoof Disentanglement Generation for Face Anti-spoofing with Depth Uncertainty Learning [54.15303628138665]
  Face anti-spoofing (FAS) plays a vital role in preventing face recognition systems from presentation attacks. Existing face anti-spoofing datasets lack diversity due to the insufficient identity and insignificant variance. We propose Dual Spoof Disentanglement Generation framework to tackle this challenge by "anti-spoofing via generation"
  arXiv  Detail & Related papers  (2021-12-01T15:36:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.