DeepRhythm: Exposing DeepFakes with Attentional Visual Heartbeat Rhythms
- URL: http://arxiv.org/abs/2006.07634v2
- Date: Wed, 26 Aug 2020 06:45:10 GMT
- Title: DeepRhythm: Exposing DeepFakes with Attentional Visual Heartbeat Rhythms
- Authors: Hua Qi and Qing Guo and Felix Juefei-Xu and Xiaofei Xie and Lei Ma and
Wei Feng and Yang Liu and Jianjun Zhao
- Abstract summary: We propose DeepRhythm, a DeepFake detection technique that exposes DeepFakes by monitoring the heartbeat rhythms.
In this work, we propose DeepRhythm, a DeepFake detection technique that exposes DeepFakes by monitoring the heartbeat rhythms.
- Score: 28.470194397110607
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: As the GAN-based face image and video generation techniques, widely known as
DeepFakes, have become more and more matured and realistic, there comes a
pressing and urgent demand for effective DeepFakes detectors. Motivated by the
fact that remote visual photoplethysmography (PPG) is made possible by
monitoring the minuscule periodic changes of skin color due to blood pumping
through the face, we conjecture that normal heartbeat rhythms found in the real
face videos will be disrupted or even entirely broken in a DeepFake video,
making it a potentially powerful indicator for DeepFake detection. In this
work, we propose DeepRhythm, a DeepFake detection technique that exposes
DeepFakes by monitoring the heartbeat rhythms. DeepRhythm utilizes
dual-spatial-temporal attention to adapt to dynamically changing face and fake
types. Extensive experiments on FaceForensics++ and DFDC-preview datasets have
confirmed our conjecture and demonstrated not only the effectiveness, but also
the generalization capability of \emph{DeepRhythm} over different datasets by
various DeepFakes generation techniques and multifarious challenging
degradations.
Related papers
- DeepFidelity: Perceptual Forgery Fidelity Assessment for Deepfake
Detection [67.3143177137102]
Deepfake detection refers to detecting artificially generated or edited faces in images or videos.
We propose a novel Deepfake detection framework named DeepFidelity to adaptively distinguish real and fake faces.
arXiv Detail & Related papers (2023-12-07T07:19:45Z) - Deep Convolutional Pooling Transformer for Deepfake Detection [54.10864860009834]
We propose a deep convolutional Transformer to incorporate decisive image features both locally and globally.
Specifically, we apply convolutional pooling and re-attention to enrich the extracted features and enhance efficacy.
The proposed solution consistently outperforms several state-of-the-art baselines on both within- and cross-dataset experiments.
arXiv Detail & Related papers (2022-09-12T15:05:41Z) - Detecting Deepfake by Creating Spatio-Temporal Regularity Disruption [94.5031244215761]
We propose to boost the generalization of deepfake detection by distinguishing the "regularity disruption" that does not appear in real videos.
Specifically, by carefully examining the spatial and temporal properties, we propose to disrupt a real video through a Pseudo-fake Generator.
Such practice allows us to achieve deepfake detection without using fake videos and improves the generalization ability in a simple and efficient manner.
arXiv Detail & Related papers (2022-07-21T10:42:34Z) - Restricted Black-box Adversarial Attack Against DeepFake Face Swapping [70.82017781235535]
We introduce a practical adversarial attack that does not require any queries to the facial image forgery model.
Our method is built on a substitute model persuing for face reconstruction and then transfers adversarial examples from the substitute model directly to inaccessible black-box DeepFake models.
arXiv Detail & Related papers (2022-04-26T14:36:06Z) - FakeTransformer: Exposing Face Forgery From Spatial-Temporal
Representation Modeled By Facial Pixel Variations [8.194624568473126]
Face forgery can attack any target, which poses a new threat to personal privacy and property security.
Inspired by the fact that the spatial coherence and temporal consistency of physiological signal are destroyed in the generated content, we attempt to find inconsistent patterns that can distinguish between real videos and synthetic videos.
arXiv Detail & Related papers (2021-11-15T08:44:52Z) - DeepFakes: Detecting Forged and Synthetic Media Content Using Machine
Learning [18.623444153774948]
The study presents challenges, research trends, and directions related to DeepFake creation and detection techniques.
The study reviews the notable research in the DeepFake domain to facilitate the development of more robust approaches that could deal with the more advance DeepFake in the future.
arXiv Detail & Related papers (2021-09-07T05:19:36Z) - DeepFake Detection with Inconsistent Head Poses: Reproducibility and
Analysis [0.0]
We analyze an existing DeepFake detection technique based on head pose estimation.
Our results correct the current literature's perception of state of the art performance for DeepFake detection.
arXiv Detail & Related papers (2021-08-28T22:56:09Z) - FFR_FD: Effective and Fast Detection of DeepFakes Based on Feature Point
Defects [9.568679090566262]
We show that DeepFake faces have fewer feature points than real ones, especially in certain facial regions.
Inspired by feature point detector-descriptors to extract discriminative features at the pixel level, we propose the Fused Facial Region_Feature Descriptor (FFR_FD) for effective and fast DeepFake detection.
arXiv Detail & Related papers (2021-07-05T13:35:39Z) - Beyond the Spectrum: Detecting Deepfakes via Re-Synthesis [69.09526348527203]
Deep generative models have led to highly realistic media, known as deepfakes, that are commonly indistinguishable from real to human eyes.
We propose a novel fake detection that is designed to re-synthesize testing images and extract visual cues for detection.
We demonstrate the improved effectiveness, cross-GAN generalization, and robustness against perturbations of our approach in a variety of detection scenarios.
arXiv Detail & Related papers (2021-05-29T21:22:24Z) - M2TR: Multi-modal Multi-scale Transformers for Deepfake Detection [74.19291916812921]
forged images generated by Deepfake techniques pose a serious threat to the trustworthiness of digital information.
In this paper, we aim to capture the subtle manipulation artifacts at different scales for Deepfake detection.
We introduce a high-quality Deepfake dataset, SR-DF, which consists of 4,000 DeepFake videos generated by state-of-the-art face swapping and facial reenactment methods.
arXiv Detail & Related papers (2021-04-20T05:43:44Z) - DeepFakesON-Phys: DeepFakes Detection based on Heart Rate Estimation [25.413558889761127]
This work introduces a novel DeepFake detection framework based on physiological measurement.
In particular, we consider methods looking for subtle color changes in the human skin, revealing the presence of human tissues under blood.
The proposed fake detector named DeepFakesONPhys uses a Convolutional Attention Network (CAN), which extracts spatial and temporal information from video frames.
arXiv Detail & Related papers (2020-10-01T13:37:58Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.