Identifying Rhythmic Patterns for Face Forgery Detection and Categorization

• URL: http://arxiv.org/abs/2207.01199v1
• Date: Mon, 4 Jul 2022 04:57:06 GMT
• Title: Identifying Rhythmic Patterns for Face Forgery Detection and Categorization
• Authors: Jiahao Liang, Weihong Deng
• Abstract summary: We propose a framework for face forgery detection and categorization consisting of: 1) a Spatial-Temporal Filtering Network (STFNet) for PPG signals, and 2) a Spatial-Temporal Interaction Network (STINet) for constraint and interaction of PPG signals. With insight into the generation of forgery methods, we further propose intra-source and inter-source blending to boost the performance of the framework.
• Score: 46.21354355137544
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With the emergence of GAN, face forgery technologies have been heavily abused. Achieving accurate face forgery detection is imminent. Inspired by remote photoplethysmography (rPPG) that PPG signal corresponds to the periodic change of skin color caused by heartbeat in face videos, we observe that despite the inevitable loss of PPG signal during the forgery process, there is still a mixture of PPG signals in the forgery video with a unique rhythmic pattern depending on its generation method. Motivated by this key observation, we propose a framework for face forgery detection and categorization consisting of: 1) a Spatial-Temporal Filtering Network (STFNet) for PPG signals filtering, and 2) a Spatial-Temporal Interaction Network (STINet) for constraint and interaction of PPG signals. Moreover, with insight into the generation of forgery methods, we further propose intra-source and inter-source blending to boost the performance of the framework. Overall, extensive experiments have proved the superiority of our method.

Related papers

• CodePhys: Robust Video-based Remote Physiological Measurement through Latent Codebook Querying [26.97093819822487]
  Remote photoplethysmography aims to measure non-contact physiological signals from facial videos. Most existing methods directly extract video-based r features by designing neural networks for heart rate estimation. Recent methods are easily affected by interference and degradation, resulting in noisy r signals. We propose a novel method named CodePhys, which innovatively treats r measurement as a code task in a noise-free proxy space.
  arXiv  Detail & Related papers  (2025-02-11T13:05:42Z)
• Partitioning Message Passing for Graph Fraud Detection [57.928658584067556]
  Label imbalance and homophily-heterophily mixture are the fundamental problems encountered when applying Graph Neural Networks (GNNs) to Graph Fraud Detection (GFD) tasks. Existing GNN-based GFD models are designed to augment graph structure to accommodate the inductive bias of GNNs towards homophily. In our work, we argue that the key to applying GNNs for GFD is not to exclude but to em distinguish neighbors with different labels.
  arXiv  Detail & Related papers  (2024-11-16T11:30:53Z)
• DD-rPPGNet: De-interfering and Descriptive Feature Learning for Unsupervised rPPG Estimation [8.901227918730562]
  Photoplethysvolution (rmography) aims to measure physiological signals and Heart Rate (HR) from facial videos. Recent unsupervised r estimation methods have shown promising potential in estimating r signals from facial regions without relying on ground truth r signals. We propose a novel Deinterfered and Descriptive r Estimation Network (DD-rNet) to eliminate the interference within r features for learning genuine r signals.
  arXiv  Detail & Related papers  (2024-07-31T07:43:58Z)
• UniForensics: Face Forgery Detection via General Facial Representation [60.5421627990707]
  High-level semantic features are less susceptible to perturbations and not limited to forgery-specific artifacts, thus having stronger generalization. We introduce UniForensics, a novel deepfake detection framework that leverages a transformer-based video network, with a meta-functional face classification for enriched facial representation.
  arXiv  Detail & Related papers  (2024-07-26T20:51:54Z)
• rFaceNet: An End-to-End Network for Enhanced Physiological Signal Extraction through Identity-Specific Facial Contours [11.050311824021733]
  Remote photoplethysmography (r) technique extracts blood volume pulse (BVP) signals from subtle pixel changes in video frames. This study introduces rFaceNet, an advanced rFaceNet method that enhances the extraction of facial BVP signals with a focus on facial contours.
  arXiv  Detail & Related papers  (2024-03-14T02:11:16Z)
• Mask Attack Detection Using Vascular-weighted Motion-robust rPPG Signals [21.884783786547782]
  R-based face anti-spoofing methods often suffer from performance degradation due to unstable face alignment in the video sequence. A landmark-anchored face stitching method is proposed to align the faces robustly and precisely at the pixel-wise level by using both SIFT keypoints and facial landmarks. A lightweight EfficientNet with a Gated Recurrent Unit (GRU) is designed to extract both spatial and temporal features for classification.
  arXiv  Detail & Related papers  (2023-05-25T11:22:17Z)
• Benchmarking Joint Face Spoofing and Forgery Detection with Visual and Physiological Cues [81.15465149555864]
  We establish the first joint face spoofing and detection benchmark using both visual appearance and physiological r cues. To enhance the r periodicity discrimination, we design a two-branch physiological network using both facial powerful rtemporal signal map and its continuous wavelet transformed counterpart as inputs.
  arXiv  Detail & Related papers  (2022-08-10T15:41:48Z)
• Consistency Regularization for Deep Face Anti-Spoofing [69.70647782777051]
  Face anti-spoofing (FAS) plays a crucial role in securing face recognition systems. Motivated by this exciting observation, we conjecture that encouraging feature consistency of different views may be a promising way to boost FAS models. We enhance both Embedding-level and Prediction-level Consistency Regularization (EPCR) in FAS.
  arXiv  Detail & Related papers  (2021-11-24T08:03:48Z)
• Heterogeneous Face Frontalization via Domain Agnostic Learning [74.86585699909459]
  We propose a domain agnostic learning-based generative adversarial network (DAL-GAN) which can synthesize frontal views in the visible domain from thermal faces with pose variations. DAL-GAN consists of a generator with an auxiliary classifier and two discriminators which capture both local and global texture discriminations for better synthesis.
  arXiv  Detail & Related papers  (2021-07-17T20:41:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.