A Convolutional LSTM based Residual Network for Deepfake Video Detection

• URL: http://arxiv.org/abs/2009.07480v1
• Date: Wed, 16 Sep 2020 05:57:06 GMT
• Title: A Convolutional LSTM based Residual Network for Deepfake Video Detection
• Authors: Shahroz Tariq, Sangyup Lee and Simon S. Woo
• Abstract summary: We develop a Convolutional LSTM based Residual Network (CLRNet) to detect deepfake videos. We also propose a transfer learning-based approach to generalize different deepfake methods.
• Score: 23.275080108063406
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In recent years, deep learning-based video manipulation methods have become widely accessible to masses. With little to no effort, people can easily learn how to generate deepfake videos with only a few victims or target images. This creates a significant social problem for everyone whose photos are publicly available on the Internet, especially on social media websites. Several deep learning-based detection methods have been developed to identify these deepfakes. However, these methods lack generalizability, because they perform well only for a specific type of deepfake method. Therefore, those methods are not transferable to detect other deepfake methods. Also, they do not take advantage of the temporal information of the video. In this paper, we addressed these limitations. We developed a Convolutional LSTM based Residual Network (CLRNet), which takes a sequence of consecutive images as an input from a video to learn the temporal information that helps in detecting unnatural looking artifacts that are present between frames of deepfake videos. We also propose a transfer learning-based approach to generalize different deepfake methods. Through rigorous experimentations using the FaceForensics++ dataset, we showed that our method outperforms five of the previously proposed state-of-the-art deepfake detection methods by better generalizing at detecting different deepfake methods using the same model.

Related papers

• Deepfake detection in videos with multiple faces using geometric-fakeness features [79.16635054977068]
  Deepfakes of victims or public figures can be used by fraudsters for blackmailing, extorsion and financial fraud. In our research we propose to use geometric-fakeness features (GFF) that characterize a dynamic degree of a face presence in a video. We employ our approach to analyze videos with multiple faces that are simultaneously present in a video.
  arXiv  Detail & Related papers  (2024-10-10T13:10:34Z)
• Undercover Deepfakes: Detecting Fake Segments in Videos [1.2609216345578933]
  deepfake generation is a new paradigm of deepfakes which are mostly real videos altered slightly to distort the truth. In this paper, we present a deepfake detection method that can address this issue by performing deepfake prediction at the frame and video levels. In particular, the paradigm we address will form a powerful tool for the moderation of deepfakes, where human oversight can be better targeted to the parts of videos suspected of being deepfakes.
  arXiv  Detail & Related papers  (2023-05-11T04:43:10Z)
• FakeOut: Leveraging Out-of-domain Self-supervision for Multi-modal Video Deepfake Detection [10.36919027402249]
  Synthetic videos of speaking humans can be used to spread misinformation in a convincing manner. FakeOut is a novel approach that relies on multi-modal data throughout both the pre-training phase and the adaption phase. Our method achieves state-of-the-art results in cross-dataset generalization on audio-visual datasets.
  arXiv  Detail & Related papers  (2022-12-01T18:56:31Z)
• Deep Convolutional Pooling Transformer for Deepfake Detection [54.10864860009834]
  We propose a deep convolutional Transformer to incorporate decisive image features both locally and globally. Specifically, we apply convolutional pooling and re-attention to enrich the extracted features and enhance efficacy. The proposed solution consistently outperforms several state-of-the-art baselines on both within- and cross-dataset experiments.
  arXiv  Detail & Related papers  (2022-09-12T15:05:41Z)
• Audio-Visual Person-of-Interest DeepFake Detection [77.04789677645682]
  The aim of this work is to propose a deepfake detector that can cope with the wide variety of manipulation methods and scenarios encountered in the real world. We leverage a contrastive learning paradigm to learn the moving-face and audio segment embeddings that are most discriminative for each identity. Our method can detect both single-modality (audio-only, video-only) and multi-modality (audio-video) attacks, and is robust to low-quality or corrupted videos.
  arXiv  Detail & Related papers  (2022-04-06T20:51:40Z)
• One Detector to Rule Them All: Towards a General Deepfake Attack Detection Framework [19.762839181838388]
  We introduce a Convolutional LSTM-based Residual Network (CLRNet) to better cope with unknown and unseen deepfakes. Our CLRNet model demonstrated that it generalizes well against high-quality DFW videos by achieving 93.86% detection accuracy.
  arXiv  Detail & Related papers  (2021-05-01T08:02:59Z)
• WildDeepfake: A Challenging Real-World Dataset for Deepfake Detection [82.42495493102805]
  We introduce a new dataset WildDeepfake which consists of 7,314 face sequences extracted from 707 deepfake videos collected completely from the internet. We conduct a systematic evaluation of a set of baseline detection networks on both existing and our WildDeepfake datasets, and show that WildDeepfake is indeed a more challenging dataset, where the detection performance can decrease drastically.
  arXiv  Detail & Related papers  (2021-01-05T11:10:32Z)
• Identity-Driven DeepFake Detection [91.0504621868628]
  Identity-Driven DeepFake Detection takes as input the suspect image/video as well as the target identity information. We output a decision on whether the identity in the suspect image/video is the same as the target identity. We present a simple identity-based detection algorithm called the OuterFace, which may serve as a baseline for further research.
  arXiv  Detail & Related papers  (2020-12-07T18:59:08Z)
• Deepfake Detection using Spatiotemporal Convolutional Networks [0.0]
  Deepfake detection methods only use individual frames and therefore fail to learn from temporal information. We created a benchmark of performance using Celeb-DF dataset. Our methods outperformed state-of-theart frame-based detection methods.
  arXiv  Detail & Related papers  (2020-06-26T01:32:31Z)
• Deepfake Video Forensics based on Transfer Learning [0.0]
  "Deepfake" can create fake images and videos that humans cannot differentiate from the genuine ones. This paper details retraining the image classification models to apprehend the features from each deepfake video frames. When checking Deepfake videos, this technique received more than 87 per cent accuracy.
  arXiv  Detail & Related papers  (2020-04-29T13:21:28Z)
• Emotions Don't Lie: An Audio-Visual Deepfake Detection Method Using Affective Cues [75.1731999380562]
  We present a learning-based method for detecting real and fake deepfake multimedia content. We extract and analyze the similarity between the two audio and visual modalities from within the same video. We compare our approach with several SOTA deepfake detection methods and report per-video AUC of 84.4% on the DFDC and 96.6% on the DF-TIMIT datasets.
  arXiv  Detail & Related papers  (2020-03-14T22:07:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.