My Face Is Mine, Not Yours: Facial Protection Against Diffusion Model Face Swapping

• URL: http://arxiv.org/abs/2505.15336v1
• Date: Wed, 21 May 2025 10:07:46 GMT
• Title: My Face Is Mine, Not Yours: Facial Protection Against Diffusion Model Face Swapping
• Authors: Hon Ming Yam, Zhongliang Guo, Chun Pong Lau,
• Abstract summary: Diffusion-based deepfake technologies pose significant risks for unauthorized and unethical facial image manipulation.<n>This paper introduces a novel proactive defense strategy through adversarial attacks that preemptively protect facial images.
• Score: 1.338174941551702
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The proliferation of diffusion-based deepfake technologies poses significant risks for unauthorized and unethical facial image manipulation. While traditional countermeasures have primarily focused on passive detection methods, this paper introduces a novel proactive defense strategy through adversarial attacks that preemptively protect facial images from being exploited by diffusion-based deepfake systems. Existing adversarial protection methods predominantly target conventional generative architectures (GANs, AEs, VAEs) and fail to address the unique challenges presented by diffusion models, which have become the predominant framework for high-quality facial deepfakes. Current diffusion-specific adversarial approaches are limited by their reliance on specific model architectures and weights, rendering them ineffective against the diverse landscape of diffusion-based deepfake implementations. Additionally, they typically employ global perturbation strategies that inadequately address the region-specific nature of facial manipulation in deepfakes.

Related papers

• Diffusion-based Adversarial Identity Manipulation for Facial Privacy Protection [14.797807196805607]
  Face recognition has led to serious privacy concerns due to potential unauthorized surveillance and user tracking on social networks.<n>Existing methods for enhancing privacy fail to generate natural face images that can protect facial privacy.<n>We propose DiffAIM to generate natural and highly transferable adversarial faces against malicious FR systems.
  arXiv  Detail & Related papers  (2025-04-30T13:49:59Z)
• Diffusion-Driven Universal Model Inversion Attack for Face Recognition [17.70133779192382]
  Face recognition systems convert raw images into embeddings, traditionally considered privacy-preserving.<n>Model inversion attacks pose a significant privacy threat by reconstructing private facial images.<n>We propose DiffUMI, a training-free diffusion-driven universal model inversion attack for face recognition systems.
  arXiv  Detail & Related papers  (2025-04-25T01:53:27Z)
• FaceShield: Defending Facial Image against Deepfake Threats [11.78218702283404]
  FaceShield is a proactive defense method targeting deepfakes generated by Diffusion Models (DMs)<n>Our approach consists of three main components: (i) manipulating the attention mechanism of DMs to exclude protected facial features during the denoising process, (ii) targeting prominent facial feature extraction models to enhance the robustness of our adversarial perturbations, and (iii) employing Gaussian blur and low-pass filtering techniques to improve imperceptibility while enhancing robustness against JPEG compression.
  arXiv  Detail & Related papers  (2024-12-13T07:20:35Z)
• DiffusionGuard: A Robust Defense Against Malicious Diffusion-based Image Editing [93.45507533317405]
  DiffusionGuard is a robust and effective defense method against unauthorized edits by diffusion-based image editing models. We introduce a novel objective that generates adversarial noise targeting the early stage of the diffusion process. We also introduce a mask-augmentation technique to enhance robustness against various masks during test time.
  arXiv  Detail & Related papers  (2024-10-08T05:19:19Z)
• DiffusionFake: Enhancing Generalization in Deepfake Detection via Guided Stable Diffusion [94.46904504076124]
  Deepfake technology has made face swapping highly realistic, raising concerns about the malicious use of fabricated facial content. Existing methods often struggle to generalize to unseen domains due to the diverse nature of facial manipulations. We introduce DiffusionFake, a novel framework that reverses the generative process of face forgeries to enhance the generalization of detection models.
  arXiv  Detail & Related papers  (2024-10-06T06:22:43Z)
• Principles of Designing Robust Remote Face Anti-Spoofing Systems [60.05766968805833]
  This paper sheds light on the vulnerabilities of state-of-the-art face anti-spoofing methods against digital attacks. It presents a comprehensive taxonomy of common threats encountered in face anti-spoofing systems.
  arXiv  Detail & Related papers  (2024-06-06T02:05:35Z)
• Attribute-Guided Encryption with Facial Texture Masking [64.77548539959501]
  We propose Attribute Guided Encryption with Facial Texture Masking to protect users from unauthorized facial recognition systems. Our proposed method produces more natural-looking encrypted images than state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-05-22T23:50:43Z)
• Detecting Adversarial Faces Using Only Real Face Self-Perturbations [36.26178169550577]
  Adrial attacks aim to disturb the functionality of a target system by adding specific noise to the input samples. Existing defense techniques achieve high accuracy in detecting some specific adversarial faces (adv-faces) New attack methods especially GAN-based attacks with completely different noise patterns circumvent them and reach a higher attack success rate.
  arXiv  Detail & Related papers  (2023-04-22T09:55:48Z)
• Data Forensics in Diffusion Models: A Systematic Analysis of Membership Privacy [62.16582309504159]
  We develop a systematic analysis of membership inference attacks on diffusion models and propose novel attack methods tailored to each attack scenario. Our approach exploits easily obtainable quantities and is highly effective, achieving near-perfect attack performance (>0.9 AUCROC) in realistic scenarios.
  arXiv  Detail & Related papers  (2023-02-15T17:37:49Z)
• Restricted Black-box Adversarial Attack Against DeepFake Face Swapping [70.82017781235535]
  We introduce a practical adversarial attack that does not require any queries to the facial image forgery model. Our method is built on a substitute model persuing for face reconstruction and then transfers adversarial examples from the substitute model directly to inaccessible black-box DeepFake models.
  arXiv  Detail & Related papers  (2022-04-26T14:36:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.