Exposing Deepfake with Pixel-wise AR and PPG Correlation from Faint Signals

• URL: http://arxiv.org/abs/2110.15561v1
• Date: Fri, 29 Oct 2021 06:05:52 GMT
• Title: Exposing Deepfake with Pixel-wise AR and PPG Correlation from Faint Signals
• Authors: Maoyu Mao and Jun Yang
• Abstract summary: Deepfake poses a serious threat to the reliability of judicial evidence and intellectual property protection. Existing pixel-level detection methods are unable to resist the growing realism of fake videos. We propose a scheme to expose Deepfake through faint signals hidden in face videos.
• Score: 3.0034765247774864
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deepfake poses a serious threat to the reliability of judicial evidence and intellectual property protection. In spite of an urgent need for Deepfake identification, existing pixel-level detection methods are increasingly unable to resist the growing realism of fake videos and lack generalization. In this paper, we propose a scheme to expose Deepfake through faint signals hidden in face videos. This scheme extracts two types of minute information hidden between face pixels-photoplethysmography (PPG) features and auto-regressive (AR) features, which are used as the basis for forensics in the temporal and spatial domains, respectively. According to the principle of PPG, tracking the absorption of light by blood cells allows remote estimation of the temporal domains heart rate (HR) of face video, and irregular HR fluctuations can be seen as traces of tampering. On the other hand, AR coefficients are able to reflect the inter-pixel correlation, and can also reflect the traces of smoothing caused by up-sampling in the process of generating fake faces. Furthermore, the scheme combines asymmetric convolution block (ACBlock)-based improved densely connected networks (DenseNets) to achieve face video authenticity forensics. Its asymmetric convolutional structure enhances the robustness of network to the input feature image upside-down and left-right flipping, so that the sequence of feature stitching does not affect detection results. Simulation results show that our proposed scheme provides more accurate authenticity detection results on multiple deep forgery datasets and has better generalization compared to the benchmark strategy.

Related papers

• GRACE: Graph-Regularized Attentive Convolutional Entanglement with Laplacian Smoothing for Robust DeepFake Video Detection [7.591187423217017]
  This paper introduces a novel method for robust DeepFake video detection based on graph convolutional network with graph Laplacian. The proposed method delivers state-of-the-art performance in DeepFake video detection under noisy face sequences.
  arXiv  Detail & Related papers  (2024-06-28T14:17:16Z)
• DeepFidelity: Perceptual Forgery Fidelity Assessment for Deepfake Detection [67.3143177137102]
  Deepfake detection refers to detecting artificially generated or edited faces in images or videos. We propose a novel Deepfake detection framework named DeepFidelity to adaptively distinguish real and fake faces.
  arXiv  Detail & Related papers  (2023-12-07T07:19:45Z)
• MMNet: Multi-Collaboration and Multi-Supervision Network for Sequential Deepfake Detection [81.59191603867586]
  Sequential deepfake detection aims to identify forged facial regions with the correct sequence for recovery. The recovery of forged images requires knowledge of the manipulation model to implement inverse transformations. We propose Multi-Collaboration and Multi-Supervision Network (MMNet) that handles various spatial scales and sequential permutations in forged face images.
  arXiv  Detail & Related papers  (2023-07-06T02:32:08Z)
• Mask Attack Detection Using Vascular-weighted Motion-robust rPPG Signals [21.884783786547782]
  R-based face anti-spoofing methods often suffer from performance degradation due to unstable face alignment in the video sequence. A landmark-anchored face stitching method is proposed to align the faces robustly and precisely at the pixel-wise level by using both SIFT keypoints and facial landmarks. A lightweight EfficientNet with a Gated Recurrent Unit (GRU) is designed to extract both spatial and temporal features for classification.
  arXiv  Detail & Related papers  (2023-05-25T11:22:17Z)
• Deep Convolutional Pooling Transformer for Deepfake Detection [54.10864860009834]
  We propose a deep convolutional Transformer to incorporate decisive image features both locally and globally. Specifically, we apply convolutional pooling and re-attention to enrich the extracted features and enhance efficacy. The proposed solution consistently outperforms several state-of-the-art baselines on both within- and cross-dataset experiments.
  arXiv  Detail & Related papers  (2022-09-12T15:05:41Z)
• Dual Spoof Disentanglement Generation for Face Anti-spoofing with Depth Uncertainty Learning [54.15303628138665]
  Face anti-spoofing (FAS) plays a vital role in preventing face recognition systems from presentation attacks. Existing face anti-spoofing datasets lack diversity due to the insufficient identity and insignificant variance. We propose Dual Spoof Disentanglement Generation framework to tackle this challenge by "anti-spoofing via generation"
  arXiv  Detail & Related papers  (2021-12-01T15:36:59Z)
• Deep Face Fuzzy Vault: Implementation and Performance [5.251555525361623]
  Unlinkable improved deep face fuzzy vault-based template protection scheme is presented. It provides privacy protection of facial reference data as well as digital key derivation from face.
  arXiv  Detail & Related papers  (2021-02-04T07:37:23Z)
• Fake face detection via adaptive manipulation traces extraction network [9.892936175042939]
  We propose an adaptive manipulation traces extraction network (AMTEN) to suppress image content and highlight manipulation traces. AMTEN exploits an adaptive convolution layer to predict manipulation traces in the image, which are reused in subsequent layers to maximize manipulation artifacts. When detecting fake face images generated by various FIM techniques, AMTENnet achieves an average accuracy up to 98.52%, which outperforms the state-of-the-art works.
  arXiv  Detail & Related papers  (2020-05-11T09:16:39Z)
• Deep Spatial Gradient and Temporal Depth Learning for Face Anti-spoofing [61.82466976737915]
  Depth supervised learning has been proven as one of the most effective methods for face anti-spoofing. We propose a new approach to detect presentation attacks from multiple frames based on two insights. The proposed approach achieves state-of-the-art results on five benchmark datasets.
  arXiv  Detail & Related papers  (2020-03-18T06:11:20Z)
• Face Anti-Spoofing by Learning Polarization Cues in a Real-World Scenario [50.36920272392624]
  Face anti-spoofing is the key to preventing security breaches in biometric recognition applications. Deep learning method using RGB and infrared images demands a large amount of training data for new attacks. We present a face anti-spoofing method in a real-world scenario by automatic learning the physical characteristics in polarization images of a real face.
  arXiv  Detail & Related papers  (2020-03-18T03:04:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.