Identifying Rhythmic Patterns for Face Forgery Detection and Categorization

• URL: http://arxiv.org/abs/2207.01199v1
• Date: Mon, 4 Jul 2022 04:57:06 GMT
• Title: Identifying Rhythmic Patterns for Face Forgery Detection and Categorization
• Authors: Jiahao Liang, Weihong Deng
• Abstract summary: We propose a framework for face forgery detection and categorization consisting of: 1) a Spatial-Temporal Filtering Network (STFNet) for PPG signals, and 2) a Spatial-Temporal Interaction Network (STINet) for constraint and interaction of PPG signals. With insight into the generation of forgery methods, we further propose intra-source and inter-source blending to boost the performance of the framework.
• Score: 46.21354355137544
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With the emergence of GAN, face forgery technologies have been heavily abused. Achieving accurate face forgery detection is imminent. Inspired by remote photoplethysmography (rPPG) that PPG signal corresponds to the periodic change of skin color caused by heartbeat in face videos, we observe that despite the inevitable loss of PPG signal during the forgery process, there is still a mixture of PPG signals in the forgery video with a unique rhythmic pattern depending on its generation method. Motivated by this key observation, we propose a framework for face forgery detection and categorization consisting of: 1) a Spatial-Temporal Filtering Network (STFNet) for PPG signals filtering, and 2) a Spatial-Temporal Interaction Network (STINet) for constraint and interaction of PPG signals. Moreover, with insight into the generation of forgery methods, we further propose intra-source and inter-source blending to boost the performance of the framework. Overall, extensive experiments have proved the superiority of our method.

Related papers

• DD-rPPGNet: De-interfering and Descriptive Feature Learning for Unsupervised rPPG Estimation [8.901227918730562]
  Photoplethysvolution (rmography) aims to measure physiological signals and Heart Rate (HR) from facial videos. Recent unsupervised r estimation methods have shown promising potential in estimating r signals from facial regions without relying on ground truth r signals. We propose a novel Deinterfered and Descriptive r Estimation Network (DD-rNet) to eliminate the interference within r features for learning genuine r signals.
  arXiv  Detail & Related papers  (2024-07-31T07:43:58Z)
• UniForensics: Face Forgery Detection via General Facial Representation [60.5421627990707]
  High-level semantic features are less susceptible to perturbations and not limited to forgery-specific artifacts, thus having stronger generalization. We introduce UniForensics, a novel deepfake detection framework that leverages a transformer-based video network, with a meta-functional face classification for enriched facial representation.
  arXiv  Detail & Related papers  (2024-07-26T20:51:54Z)
• rFaceNet: An End-to-End Network for Enhanced Physiological Signal Extraction through Identity-Specific Facial Contours [11.050311824021733]
  Remote photoplethysmography (r) technique extracts blood volume pulse (BVP) signals from subtle pixel changes in video frames. This study introduces rFaceNet, an advanced rFaceNet method that enhances the extraction of facial BVP signals with a focus on facial contours.
  arXiv  Detail & Related papers  (2024-03-14T02:11:16Z)
• Forgery-aware Adaptive Transformer for Generalizable Synthetic Image Detection [106.39544368711427]
  We study the problem of generalizable synthetic image detection, aiming to detect forgery images from diverse generative methods. We present a novel forgery-aware adaptive transformer approach, namely FatFormer. Our approach tuned on 4-class ProGAN data attains an average of 98% accuracy to unseen GANs, and surprisingly generalizes to unseen diffusion models with 95% accuracy.
  arXiv  Detail & Related papers  (2023-12-27T17:36:32Z)
• Mask Attack Detection Using Vascular-weighted Motion-robust rPPG Signals [21.884783786547782]
  R-based face anti-spoofing methods often suffer from performance degradation due to unstable face alignment in the video sequence. A landmark-anchored face stitching method is proposed to align the faces robustly and precisely at the pixel-wise level by using both SIFT keypoints and facial landmarks. A lightweight EfficientNet with a Gated Recurrent Unit (GRU) is designed to extract both spatial and temporal features for classification.
  arXiv  Detail & Related papers  (2023-05-25T11:22:17Z)
• Benchmarking Joint Face Spoofing and Forgery Detection with Visual and Physiological Cues [81.15465149555864]
  We establish the first joint face spoofing and detection benchmark using both visual appearance and physiological r cues. To enhance the r periodicity discrimination, we design a two-branch physiological network using both facial powerful rtemporal signal map and its continuous wavelet transformed counterpart as inputs.
  arXiv  Detail & Related papers  (2022-08-10T15:41:48Z)
• Consistency Regularization for Deep Face Anti-Spoofing [69.70647782777051]
  Face anti-spoofing (FAS) plays a crucial role in securing face recognition systems. Motivated by this exciting observation, we conjecture that encouraging feature consistency of different views may be a promising way to boost FAS models. We enhance both Embedding-level and Prediction-level Consistency Regularization (EPCR) in FAS.
  arXiv  Detail & Related papers  (2021-11-24T08:03:48Z)
• Motion Artifact Reduction In Photoplethysmography For Reliable Signal Selection [5.264561559435017]
  Photoplethysmography ( PPG) is a non-invasive and economical technique to extract vital signs of the human body. It is sensitive to motion which can corrupt the signal's quality. It is valuable to collect realistic PPG signals while performing Activities of Daily Living (ADL) to develop practical signal denoising and analysis methods.
  arXiv  Detail & Related papers  (2021-09-06T21:53:56Z)
• Heterogeneous Face Frontalization via Domain Agnostic Learning [74.86585699909459]
  We propose a domain agnostic learning-based generative adversarial network (DAL-GAN) which can synthesize frontal views in the visible domain from thermal faces with pose variations. DAL-GAN consists of a generator with an auxiliary classifier and two discriminators which capture both local and global texture discriminations for better synthesis.
  arXiv  Detail & Related papers  (2021-07-17T20:41:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.