ID-Reveal: Identity-aware DeepFake Video Detection
- URL: http://arxiv.org/abs/2012.02512v2
- Date: Fri, 23 Apr 2021 07:43:18 GMT
- Title: ID-Reveal: Identity-aware DeepFake Video Detection
- Authors: Davide Cozzolino and Andreas R\"ossler and Justus Thies and Matthias
Nie{\ss}ner and Luisa Verdoliva
- Abstract summary: ID-Reveal is a new approach that learns temporal facial features, specific of how a person moves while talking.
We do not need any training data of fakes, but only train on real videos.
We obtain an average improvement of more than 15% in terms of accuracy for facial reenactment on high compressed videos.
- Score: 24.79483180234883
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: A major challenge in DeepFake forgery detection is that state-of-the-art
algorithms are mostly trained to detect a specific fake method. As a result,
these approaches show poor generalization across different types of facial
manipulations, e.g., from face swapping to facial reenactment. To this end, we
introduce ID-Reveal, a new approach that learns temporal facial features,
specific of how a person moves while talking, by means of metric learning
coupled with an adversarial training strategy. The advantage is that we do not
need any training data of fakes, but only train on real videos. Moreover, we
utilize high-level semantic features, which enables robustess to widespread and
disruptive forms of post-processing. We perform a thorough experimental
analysis on several publicly available benchmarks. Compared to state of the
art, our method improves generalization and is more robust to low-quality
videos, that are usually spread over social networks. In particular, we obtain
an average improvement of more than 15% in terms of accuracy for facial
reenactment on high compressed videos.
Related papers
- Deepfake detection in videos with multiple faces using geometric-fakeness features [79.16635054977068]
Deepfakes of victims or public figures can be used by fraudsters for blackmailing, extorsion and financial fraud.
In our research we propose to use geometric-fakeness features (GFF) that characterize a dynamic degree of a face presence in a video.
We employ our approach to analyze videos with multiple faces that are simultaneously present in a video.
arXiv Detail & Related papers (2024-10-10T13:10:34Z) - Mover: Mask and Recovery based Facial Part Consistency Aware Method for
Deepfake Video Detection [33.29744034340998]
Mover is a new Deepfake detection model that exploits unspecific facial part inconsistencies.
We propose a novel model with dual networks that utilize the pretrained encoder and masked autoencoder.
Our experiments on standard benchmarks demonstrate that Mover is highly effective.
arXiv Detail & Related papers (2023-03-03T06:57:22Z) - Audio-Visual Person-of-Interest DeepFake Detection [77.04789677645682]
The aim of this work is to propose a deepfake detector that can cope with the wide variety of manipulation methods and scenarios encountered in the real world.
We leverage a contrastive learning paradigm to learn the moving-face and audio segment embeddings that are most discriminative for each identity.
Our method can detect both single-modality (audio-only, video-only) and multi-modality (audio-video) attacks, and is robust to low-quality or corrupted videos.
arXiv Detail & Related papers (2022-04-06T20:51:40Z) - Self-supervised Transformer for Deepfake Detection [112.81127845409002]
Deepfake techniques in real-world scenarios require stronger generalization abilities of face forgery detectors.
Inspired by transfer learning, neural networks pre-trained on other large-scale face-related tasks may provide useful features for deepfake detection.
In this paper, we propose a self-supervised transformer based audio-visual contrastive learning method.
arXiv Detail & Related papers (2022-03-02T17:44:40Z) - Leveraging Real Talking Faces via Self-Supervision for Robust Forgery
Detection [112.96004727646115]
We develop a method to detect face-manipulated videos using real talking faces.
We show that our method achieves state-of-the-art performance on cross-manipulation generalisation and robustness experiments.
Our results suggest that leveraging natural and unlabelled videos is a promising direction for the development of more robust face forgery detectors.
arXiv Detail & Related papers (2022-01-18T17:14:54Z) - Improving DeepFake Detection Using Dynamic Face Augmentation [0.8793721044482612]
Most publicly available DeepFake detection datasets have limited variations.
Deep neural networks tend to overfit to the facial features instead of learning to detect manipulation features of DeepFake content.
We introduce Face-Cutout, a data augmentation method for training Convolutional Neural Networks (CNN) to improve DeepFake detection.
arXiv Detail & Related papers (2021-02-18T20:25:45Z) - Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery
Detection [118.37239586697139]
LipForensics is a detection approach capable of both generalising manipulations and withstanding various distortions.
It consists in first pretraining a-temporal network to perform visual speech recognition (lipreading)
A temporal network is subsequently finetuned on fixed mouth embeddings of real and forged data in order to detect fake videos based on mouth movements without over-fitting to low-level, manipulation-specific artefacts.
arXiv Detail & Related papers (2020-12-14T15:53:56Z) - A Convolutional LSTM based Residual Network for Deepfake Video Detection [23.275080108063406]
We develop a Convolutional LSTM based Residual Network (CLRNet) to detect deepfake videos.
We also propose a transfer learning-based approach to generalize different deepfake methods.
arXiv Detail & Related papers (2020-09-16T05:57:06Z) - Detecting Face2Face Facial Reenactment in Videos [76.9573023955201]
This research proposes a learning-based algorithm for detecting reenactment based alterations.
The proposed algorithm uses a multi-stream network that learns regional artifacts and provides a robust performance at various compression levels.
The results show state-of-the-art classification accuracy of 99.96%, 99.10%, and 91.20% for no, easy, and hard compression factors, respectively.
arXiv Detail & Related papers (2020-01-21T11:03:50Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.