DeepRhythm: Exposing DeepFakes with Attentional Visual Heartbeat Rhythms
- URL: http://arxiv.org/abs/2006.07634v2
- Date: Wed, 26 Aug 2020 06:45:10 GMT
- Title: DeepRhythm: Exposing DeepFakes with Attentional Visual Heartbeat Rhythms
- Authors: Hua Qi and Qing Guo and Felix Juefei-Xu and Xiaofei Xie and Lei Ma and
Wei Feng and Yang Liu and Jianjun Zhao
- Abstract summary: We propose DeepRhythm, a DeepFake detection technique that exposes DeepFakes by monitoring the heartbeat rhythms.
In this work, we propose DeepRhythm, a DeepFake detection technique that exposes DeepFakes by monitoring the heartbeat rhythms.
- Score: 28.470194397110607
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: As the GAN-based face image and video generation techniques, widely known as
DeepFakes, have become more and more matured and realistic, there comes a
pressing and urgent demand for effective DeepFakes detectors. Motivated by the
fact that remote visual photoplethysmography (PPG) is made possible by
monitoring the minuscule periodic changes of skin color due to blood pumping
through the face, we conjecture that normal heartbeat rhythms found in the real
face videos will be disrupted or even entirely broken in a DeepFake video,
making it a potentially powerful indicator for DeepFake detection. In this
work, we propose DeepRhythm, a DeepFake detection technique that exposes
DeepFakes by monitoring the heartbeat rhythms. DeepRhythm utilizes
dual-spatial-temporal attention to adapt to dynamically changing face and fake
types. Extensive experiments on FaceForensics++ and DFDC-preview datasets have
confirmed our conjecture and demonstrated not only the effectiveness, but also
the generalization capability of \emph{DeepRhythm} over different datasets by
various DeepFakes generation techniques and multifarious challenging
degradations.
Related papers
- Deepfake detection in videos with multiple faces using geometric-fakeness features [79.16635054977068]
Deepfakes of victims or public figures can be used by fraudsters for blackmailing, extorsion and financial fraud.
In our research we propose to use geometric-fakeness features (GFF) that characterize a dynamic degree of a face presence in a video.
We employ our approach to analyze videos with multiple faces that are simultaneously present in a video.
arXiv Detail & Related papers (2024-10-10T13:10:34Z) - DiffusionFake: Enhancing Generalization in Deepfake Detection via Guided Stable Diffusion [94.46904504076124]
Deepfake technology has made face swapping highly realistic, raising concerns about the malicious use of fabricated facial content.
Existing methods often struggle to generalize to unseen domains due to the diverse nature of facial manipulations.
We introduce DiffusionFake, a novel framework that reverses the generative process of face forgeries to enhance the generalization of detection models.
arXiv Detail & Related papers (2024-10-06T06:22:43Z) - Deep Convolutional Pooling Transformer for Deepfake Detection [54.10864860009834]
We propose a deep convolutional Transformer to incorporate decisive image features both locally and globally.
Specifically, we apply convolutional pooling and re-attention to enrich the extracted features and enhance efficacy.
The proposed solution consistently outperforms several state-of-the-art baselines on both within- and cross-dataset experiments.
arXiv Detail & Related papers (2022-09-12T15:05:41Z) - Detecting Deepfake by Creating Spatio-Temporal Regularity Disruption [94.5031244215761]
We propose to boost the generalization of deepfake detection by distinguishing the "regularity disruption" that does not appear in real videos.
Specifically, by carefully examining the spatial and temporal properties, we propose to disrupt a real video through a Pseudo-fake Generator.
Such practice allows us to achieve deepfake detection without using fake videos and improves the generalization ability in a simple and efficient manner.
arXiv Detail & Related papers (2022-07-21T10:42:34Z) - Restricted Black-box Adversarial Attack Against DeepFake Face Swapping [70.82017781235535]
We introduce a practical adversarial attack that does not require any queries to the facial image forgery model.
Our method is built on a substitute model persuing for face reconstruction and then transfers adversarial examples from the substitute model directly to inaccessible black-box DeepFake models.
arXiv Detail & Related papers (2022-04-26T14:36:06Z) - DeepFakes: Detecting Forged and Synthetic Media Content Using Machine
Learning [18.623444153774948]
The study presents challenges, research trends, and directions related to DeepFake creation and detection techniques.
The study reviews the notable research in the DeepFake domain to facilitate the development of more robust approaches that could deal with the more advance DeepFake in the future.
arXiv Detail & Related papers (2021-09-07T05:19:36Z) - DeepFake Detection with Inconsistent Head Poses: Reproducibility and
Analysis [0.0]
We analyze an existing DeepFake detection technique based on head pose estimation.
Our results correct the current literature's perception of state of the art performance for DeepFake detection.
arXiv Detail & Related papers (2021-08-28T22:56:09Z) - FFR_FD: Effective and Fast Detection of DeepFakes Based on Feature Point
Defects [9.568679090566262]
We show that DeepFake faces have fewer feature points than real ones, especially in certain facial regions.
Inspired by feature point detector-descriptors to extract discriminative features at the pixel level, we propose the Fused Facial Region_Feature Descriptor (FFR_FD) for effective and fast DeepFake detection.
arXiv Detail & Related papers (2021-07-05T13:35:39Z) - Beyond the Spectrum: Detecting Deepfakes via Re-Synthesis [69.09526348527203]
Deep generative models have led to highly realistic media, known as deepfakes, that are commonly indistinguishable from real to human eyes.
We propose a novel fake detection that is designed to re-synthesize testing images and extract visual cues for detection.
We demonstrate the improved effectiveness, cross-GAN generalization, and robustness against perturbations of our approach in a variety of detection scenarios.
arXiv Detail & Related papers (2021-05-29T21:22:24Z) - DeepFakesON-Phys: DeepFakes Detection based on Heart Rate Estimation [25.413558889761127]
This work introduces a novel DeepFake detection framework based on physiological measurement.
In particular, we consider methods looking for subtle color changes in the human skin, revealing the presence of human tissues under blood.
The proposed fake detector named DeepFakesONPhys uses a Convolutional Attention Network (CAN), which extracts spatial and temporal information from video frames.
arXiv Detail & Related papers (2020-10-01T13:37:58Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.