DiffMI: Breaking Face Recognition Privacy via Diffusion-Driven Training-Free Model Inversion

• URL: http://arxiv.org/abs/2504.18015v3
• Date: Wed, 08 Oct 2025 13:46:41 GMT
• Title: DiffMI: Breaking Face Recognition Privacy via Diffusion-Driven Training-Free Model Inversion
• Authors: Hanrui Wang, Shuo Wang, Chun-Shien Lu, Isao Echizen,
• Abstract summary: Face recognition poses serious privacy risks due to reliance on immutable biometric data.<n>Model inversion attacks reveal that identity information can still be recovered, exposing critical vulnerabilities.<n>We propose DiffMI, the first diffusion-driven, training-free model inversion attack.
• Score: 16.02881487974147
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Face recognition poses serious privacy risks due to its reliance on sensitive and immutable biometric data. While modern systems mitigate privacy risks by mapping facial images to embeddings (commonly regarded as privacy-preserving), model inversion attacks reveal that identity information can still be recovered, exposing critical vulnerabilities. However, existing attacks are often computationally expensive and lack generalization, especially those requiring target-specific training. Even training-free approaches suffer from limited identity controllability, hindering faithful reconstruction of nuanced or unseen identities. In this work, we propose DiffMI, the first diffusion-driven, training-free model inversion attack. DiffMI introduces a novel pipeline combining robust latent code initialization, a ranked adversarial refinement strategy, and a statistically grounded, confidence-aware optimization objective. DiffMI applies directly to unseen target identities and face recognition models, offering greater adaptability than training-dependent approaches while significantly reducing computational overhead. Our method achieves 84.42%--92.87% attack success rates against inversion-resilient systems and outperforms the best prior training-free GAN-based approach by 4.01%--9.82%. The implementation is available at https://github.com/azrealwang/DiffMI.

Related papers

• Res-MIA: A Training-Free Resolution-Based Membership Inference Attack on Federated Learning Models [1.9336815376402718]
  Membership inference attacks pose a serious threat to the privacy of machine learning models.<n>We introduce Res-MIA, a training-free and black-box membership inference attack.<n>We evaluate the proposed attack on a federated ResNet-18 trained on CIFAR-10.
  arXiv  Detail & Related papers  (2026-01-24T08:58:39Z)
• Deep Leakage with Generative Flow Matching Denoiser [54.05993847488204]
  We introduce a new deep leakage (DL) attack that integrates a generative Flow Matching (FM) prior into the reconstruction process.<n>Our approach consistently outperforms state-of-the-art attacks across pixel-level, perceptual, and feature-based similarity metrics.
  arXiv  Detail & Related papers  (2026-01-21T14:51:01Z)
• Reminiscence Attack on Residuals: Exploiting Approximate Machine Unlearning for Privacy [18.219835803238837]
  We show that approximate unlearning algorithms fail to adequately protect the privacy of unlearned data.<n>We propose the Reminiscence Attack (ReA), which amplifies the correlation between residuals and membership privacy.<n>We develop a dual-phase approximate unlearning framework that first eliminates deep-layer unlearned data traces and then enforces convergence stability.
  arXiv  Detail & Related papers  (2025-07-28T07:12:12Z)
• Enhancing Facial Privacy Protection via Weakening Diffusion Purification [36.33027625681024]
  Social media has led to the widespread sharing of individual portrait images, which pose serious privacy risks.<n>Recent methods employ diffusion models to generate adversarial face images for privacy protection.<n>We propose learning unconditional embeddings to increase the learning capacity for adversarial modifications.<n>We integrate an identity-preserving structure to maintain structural consistency between the original and generated images.
  arXiv  Detail & Related papers  (2025-03-13T13:27:53Z)
• iFADIT: Invertible Face Anonymization via Disentangled Identity Transform [51.123936665445356]
  Face anonymization aims to conceal the visual identity of a face to safeguard the individual's privacy.<n>This paper proposes a novel framework named iFADIT, an acronym for Invertible Face Anonymization via Disentangled Identity Transform.
  arXiv  Detail & Related papers  (2025-01-08T10:08:09Z)
• Local Features Meet Stochastic Anonymization: Revolutionizing Privacy-Preserving Face Recognition for Black-Box Models [54.88064975480573]
  The task of privacy-preserving face recognition (PPFR) currently faces two major unsolved challenges.<n>By disrupting global features while enhancing local features, we achieve effective recognition even in black-box environments.<n>Our method achieves an average recognition accuracy of 94.21% on black-box models, outperforming existing methods in both privacy protection and anti-reconstruction capabilities.
  arXiv  Detail & Related papers  (2024-12-11T10:49:15Z)
• OSDFace: One-Step Diffusion Model for Face Restoration [72.5045389847792]
  Diffusion models have demonstrated impressive performance in face restoration.<n>We propose OSDFace, a novel one-step diffusion model for face restoration.<n>Results demonstrate that OSDFace surpasses current state-of-the-art (SOTA) methods in both visual quality and quantitative metrics.
  arXiv  Detail & Related papers  (2024-11-26T07:07:48Z)
• ID$^3$: Identity-Preserving-yet-Diversified Diffusion Models for Synthetic Face Recognition [60.15830516741776]
  Synthetic face recognition (SFR) aims to generate datasets that mimic the distribution of real face data. We introduce a diffusion-fueled SFR model termed $textID3$. $textID3$ employs an ID-preserving loss to generate diverse yet identity-consistent facial appearances.
  arXiv  Detail & Related papers  (2024-09-26T06:46:40Z)
• A Practical Trigger-Free Backdoor Attack on Neural Networks [33.426207982772226]
  We propose a trigger-free backdoor attack that does not require access to any training data. Specifically, we design a novel fine-tuning approach that incorporates the concept of malicious data into the concept of the attacker-specified class. The effectiveness, practicality, and stealthiness of the proposed attack are evaluated on three real-world datasets.
  arXiv  Detail & Related papers  (2024-08-21T08:53:36Z)
• Transferable Adversarial Facial Images for Privacy Protection [15.211743719312613]
  We present a novel face privacy protection scheme with improved transferability while maintain high visual quality. We first exploit global adversarial latent search to traverse the latent space of the generative model. We then introduce a key landmark regularization module to preserve the visual identity information.
  arXiv  Detail & Related papers  (2024-07-18T02:16:11Z)
• GI-NAS: Boosting Gradient Inversion Attacks Through Adaptive Neural Architecture Search [52.27057178618773]
  Gradient Inversion Attacks invert the transmitted gradients in Federated Learning (FL) systems to reconstruct the sensitive data of local clients.<n>A majority of gradient inversion methods rely heavily on explicit prior knowledge, which is often unavailable in realistic scenarios.<n>We propose Neural Architecture Search (GI-NAS), which adaptively searches the network and captures the implicit priors behind neural architectures.
  arXiv  Detail & Related papers  (2024-05-31T09:29:43Z)
• Privacy-Preserving Face Recognition in Hybrid Frequency-Color Domain [16.05230409730324]
  Face image is a sensitive biometric attribute tied to the identity information of each user. This paper proposes a hybrid frequency-color fusion approach to reduce the input dimensionality of face recognition. It has around 2.6% to 4.2% higher accuracy than the state-of-the-art in the 1:N verification scenario.
  arXiv  Detail & Related papers  (2024-01-24T11:27:32Z)
• Generalized Face Liveness Detection via De-fake Face Generator [52.23271636362843]
  Previous Face Anti-spoofing (FAS) methods face the challenge of generalizing to unseen domains.<n>We propose an Anomalous cue Guided FAS (AG-FAS) method, which can effectively leverage large-scale additional real faces.<n>Our method achieves state-of-the-art results under cross-domain evaluations with unseen scenarios and unknown presentation attacks.
  arXiv  Detail & Related papers  (2024-01-17T06:59:32Z)
• Adv-Diffusion: Imperceptible Adversarial Face Identity Attack via Latent Diffusion Model [61.53213964333474]
  We propose a unified framework Adv-Diffusion that can generate imperceptible adversarial identity perturbations in the latent space but not the raw pixel space. Specifically, we propose the identity-sensitive conditioned diffusion generative model to generate semantic perturbations in the surroundings. The designed adaptive strength-based adversarial perturbation algorithm can ensure both attack transferability and stealthiness.
  arXiv  Detail & Related papers  (2023-12-18T15:25:23Z)
• Client-side Gradient Inversion Against Federated Learning from Poisoning [59.74484221875662]
  Federated Learning (FL) enables distributed participants to train a global model without sharing data directly to a central server. Recent studies have revealed that FL is vulnerable to gradient inversion attack (GIA), which aims to reconstruct the original training samples. We propose Client-side poisoning Gradient Inversion (CGI), which is a novel attack method that can be launched from clients.
  arXiv  Detail & Related papers  (2023-09-14T03:48:27Z)
• Diff-Privacy: Diffusion-based Face Privacy Protection [58.1021066224765]
  In this paper, we propose a novel face privacy protection method based on diffusion models, dubbed Diff-Privacy. Specifically, we train our proposed multi-scale image inversion module (MSI) to obtain a set of SDM format conditional embeddings of the original image. Based on the conditional embeddings, we design corresponding embedding scheduling strategies and construct different energy functions during the denoising process to achieve anonymization and visual identity information hiding.
  arXiv  Detail & Related papers  (2023-09-11T09:26:07Z)
• Avoid Adversarial Adaption in Federated Learning by Multi-Metric Investigations [55.2480439325792]
  Federated Learning (FL) facilitates decentralized machine learning model training, preserving data privacy, lowering communication costs, and boosting model performance through diversified data sources. FL faces vulnerabilities such as poisoning attacks, undermining model integrity with both untargeted performance degradation and targeted backdoor attacks. We define a new notion of strong adaptive adversaries, capable of adapting to multiple objectives simultaneously. MESAS is the first defense robust against strong adaptive adversaries, effective in real-world data scenarios, with an average overhead of just 24.37 seconds.
  arXiv  Detail & Related papers  (2023-06-06T11:44:42Z)
• Controllable Inversion of Black-Box Face Recognition Models via Diffusion [8.620807177029892]
  We tackle the task of inverting the latent space of pre-trained face recognition models without full model access. We show that the conditional diffusion model loss naturally emerges and that we can effectively sample from the inverse distribution. Our method is the first black-box face recognition model inversion method that offers intuitive control over the generation process.
  arXiv  Detail & Related papers  (2023-03-23T03:02:09Z)
• RelaxLoss: Defending Membership Inference Attacks without Losing Utility [68.48117818874155]
  We propose a novel training framework based on a relaxed loss with a more achievable learning target. RelaxLoss is applicable to any classification model with added benefits of easy implementation and negligible overhead. Our approach consistently outperforms state-of-the-art defense mechanisms in terms of resilience against MIAs.
  arXiv  Detail & Related papers  (2022-07-12T19:34:47Z)
• Improving Transferability of Adversarial Patches on Face Recognition with Generative Models [43.51625789744288]
  We evaluate the robustness of face recognition models using adversarial patches based on transferability. We show that the gaps between the responses of substitute models and the target models dramatically decrease, exhibiting a better transferability.
  arXiv  Detail & Related papers  (2021-06-29T02:13:05Z)
• Systematic Evaluation of Privacy Risks of Machine Learning Models [41.017707772150835]
  We show that prior work on membership inference attacks may severely underestimate the privacy risks. We first propose to benchmark membership inference privacy risks by improving existing non-neural network based inference attacks. We then introduce a new approach for fine-grained privacy analysis by formulating and deriving a new metric called the privacy risk score.
  arXiv  Detail & Related papers  (2020-03-24T00:53:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.