DFREC: DeepFake Identity Recovery Based on Identity-aware Masked Autoencoder

• URL: http://arxiv.org/abs/2412.07260v1
• Date: Tue, 10 Dec 2024 07:42:02 GMT
• Title: DFREC: DeepFake Identity Recovery Based on Identity-aware Masked Autoencoder
• Authors: Peipeng Yu, Hui Gao, Zhitao Huang, Zhihua Xia, Chip-Hong Chang,
• Abstract summary: DFREC aims to recover the pair of source and target faces from a deepfake image to facilitate deepfake identity tracing. We evaluate DFREC on six different high-fidelity face-swapping attacks on FaceForensics++, CelebaFS and FFHQ-E4S datasets.
• Score: 18.03674138250793
• License:
• Abstract: Recent advances in deepfake forensics have primarily focused on improving the classification accuracy and generalization performance. Despite enormous progress in detection accuracy across a wide variety of forgery algorithms, existing algorithms lack intuitive interpretability and identity traceability to help with forensic investigation. In this paper, we introduce a novel DeepFake Identity Recovery scheme (DFREC) to fill this gap. DFREC aims to recover the pair of source and target faces from a deepfake image to facilitate deepfake identity tracing and reduce the risk of deepfake attack. It comprises three key components: an Identity Segmentation Module (ISM), a Source Identity Reconstruction Module (SIRM), and a Target Identity Reconstruction Module (TIRM). The ISM segments the input face into distinct source and target face information, and the SIRM reconstructs the source face and extracts latent target identity features with the segmented source information. The background context and latent target identity features are synergetically fused by a Masked Autoencoder in the TIRM to reconstruct the target face. We evaluate DFREC on six different high-fidelity face-swapping attacks on FaceForensics++, CelebaMegaFS and FFHQ-E4S datasets, which demonstrate its superior recovery performance over state-of-the-art deepfake recovery algorithms. In addition, DFREC is the only scheme that can recover both pristine source and target faces directly from the forgery image with high fadelity.

Related papers

• DiffusionFake: Enhancing Generalization in Deepfake Detection via Guided Stable Diffusion [94.46904504076124]
  Deepfake technology has made face swapping highly realistic, raising concerns about the malicious use of fabricated facial content. Existing methods often struggle to generalize to unseen domains due to the diverse nature of facial manipulations. We introduce DiffusionFake, a novel framework that reverses the generative process of face forgeries to enhance the generalization of detection models.
  arXiv  Detail & Related papers  (2024-10-06T06:22:43Z)
• IDRetracor: Towards Visual Forensics Against Malicious Face Swapping [30.804429527783395]
  Face swapping technique based on deepfake methods poses significant social risks to personal identity security. We propose a novel task named face retracing, which considers retracing the original target face from the given fake one via inverse mapping. We show that the IDRetracor exhibits promising retracing performance from both quantitative and qualitative perspectives.
  arXiv  Detail & Related papers  (2024-08-13T04:53:48Z)
• UniForensics: Face Forgery Detection via General Facial Representation [60.5421627990707]
  High-level semantic features are less susceptible to perturbations and not limited to forgery-specific artifacts, thus having stronger generalization. We introduce UniForensics, a novel deepfake detection framework that leverages a transformer-based video network, with a meta-functional face classification for enriched facial representation.
  arXiv  Detail & Related papers  (2024-07-26T20:51:54Z)
• Face Reconstruction Transfer Attack as Out-of-Distribution Generalization [15.258162177124317]
  We aim to reconstruct face images which are capable of transferring face attacks on unseen encoders. Inspired by its OOD nature, we propose to solve Face Reconstruction Transfer Attack (FRTA) by Averaged Latent Search and Unsupervised Validation with pseudo target (ALSUV)
  arXiv  Detail & Related papers  (2024-07-02T16:21:44Z)
• UGMAE: A Unified Framework for Graph Masked Autoencoders [67.75493040186859]
  We propose UGMAE, a unified framework for graph masked autoencoders. We first develop an adaptive feature mask generator to account for the unique significance of nodes. We then design a ranking-based structure reconstruction objective joint with feature reconstruction to capture holistic graph information.
  arXiv  Detail & Related papers  (2024-02-12T19:39:26Z)
• Cross-domain Robust Deepfake Bias Expansion Network for Face Forgery Detection [4.269822517578155]
  We introduce a Cross-Domain Robust Bias Expansion Network (BENet) to enhance face forgery detection. BENet employs an auto-encoder to reconstruct input faces, maintaining the invariance of real faces while selectively enhancing the difference between reconstructed fake faces and their original counterparts. In addition, BENet incorporates a cross-domain detector with a threshold to determine whether the sample belongs to a known distribution.
  arXiv  Detail & Related papers  (2023-10-08T11:30:22Z)
• Vec2Face-v2: Unveil Human Faces from their Blackbox Features via Attention-based Network in Face Recognition [36.23997331928846]
  We introduce a new method named Attention-based Bijective Generative Adversarial Networks in a Distillation framework (DAB-GAN) The DAB-GAN method includes a novel attention-based generative structure with the newly defined Bijective Metrics Learning approach. We have evaluated our method on the challenging face recognition databases.
  arXiv  Detail & Related papers  (2022-09-11T19:14:21Z)
• Exposing Deepfake with Pixel-wise AR and PPG Correlation from Faint Signals [3.0034765247774864]
  Deepfake poses a serious threat to the reliability of judicial evidence and intellectual property protection. Existing pixel-level detection methods are unable to resist the growing realism of fake videos. We propose a scheme to expose Deepfake through faint signals hidden in face videos.
  arXiv  Detail & Related papers  (2021-10-29T06:05:52Z)
• End2End Occluded Face Recognition by Masking Corrupted Features [82.27588990277192]
  State-of-the-art general face recognition models do not generalize well to occluded face images. This paper presents a novel face recognition method that is robust to occlusions based on a single end-to-end deep neural network. Our approach, named FROM (Face Recognition with Occlusion Masks), learns to discover the corrupted features from the deep convolutional neural networks, and clean them by the dynamically learned masks.
  arXiv  Detail & Related papers  (2021-08-21T09:08:41Z)
• Identity-Driven DeepFake Detection [91.0504621868628]
  Identity-Driven DeepFake Detection takes as input the suspect image/video as well as the target identity information. We output a decision on whether the identity in the suspect image/video is the same as the target identity. We present a simple identity-based detection algorithm called the OuterFace, which may serve as a baseline for further research.
  arXiv  Detail & Related papers  (2020-12-07T18:59:08Z)
• Deep Spatial Gradient and Temporal Depth Learning for Face Anti-spoofing [61.82466976737915]
  Depth supervised learning has been proven as one of the most effective methods for face anti-spoofing. We propose a new approach to detect presentation attacks from multiple frames based on two insights. The proposed approach achieves state-of-the-art results on five benchmark datasets.
  arXiv  Detail & Related papers  (2020-03-18T06:11:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.