A Hybrid CNN-LSTM model for Video Deepfake Detection by Leveraging Optical Flow Features

• URL: http://arxiv.org/abs/2208.00788v1
• Date: Thu, 28 Jul 2022 09:38:09 GMT
• Title: A Hybrid CNN-LSTM model for Video Deepfake Detection by Leveraging Optical Flow Features
• Authors: Pallabi Saikia, Dhwani Dholaria, Priyanka Yadav, Vaidehi Patel, Mohendra Roy
• Abstract summary: Deepfakes are the synthesized digital media in order to create ultra-realistic fake videos to trick the spectator. In this paper, we leveraged an optical flow based feature extraction approach to extract the temporal features, which are then fed to a hybrid model for classification. The hybrid model provides effective performance on open source data-sets such as, DFDC, FF++ and Celeb-DF.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deepfakes are the synthesized digital media in order to create ultra-realistic fake videos to trick the spectator. Deep generative algorithms, such as, Generative Adversarial Networks(GAN) are widely used to accomplish such tasks. This approach synthesizes pseudo-realistic contents that are very difficult to distinguish by traditional detection methods. In most cases, Convolutional Neural Network(CNN) based discriminators are being used for detecting such synthesized media. However, it emphasise primarily on the spatial attributes of individual video frames, thereby fail to learn the temporal information from their inter-frame relations. In this paper, we leveraged an optical flow based feature extraction approach to extract the temporal features, which are then fed to a hybrid model for classification. This hybrid model is based on the combination of CNN and recurrent neural network (RNN) architectures. The hybrid model provides effective performance on open source data-sets such as, DFDC, FF++ and Celeb-DF. This proposed method shows an accuracy of 66.26%, 91.21% and 79.49% in DFDC, FF++, and Celeb-DF respectively with a very reduced No of sample size of approx 100 samples(frames). This promises early detection of fake contents compared to existing modalities.

Related papers

• Neural Network Parameter Diffusion [50.85251415173792]
  Diffusion models have achieved remarkable success in image and video generation. In this work, we demonstrate that diffusion models can also. generate high-performing neural network parameters.
  arXiv  Detail & Related papers  (2024-02-20T16:59:03Z)
• CapST: An Enhanced and Lightweight Model Attribution Approach for Synthetic Videos [9.209808258321559]
  This paper investigates the model attribution problem of Deepfake videos from a recently proposed dataset, Deepfakes from Different Models (DFDM) The dataset comprises 6,450 Deepfake videos generated by five distinct models with variations in encoder, decoder, intermediate layer, input resolution, and compression ratio. Experimental results on the deepfake benchmark dataset (DFDM) demonstrate the efficacy of our proposed method, achieving up to a 4% improvement in accurately categorizing deepfake videos.
  arXiv  Detail & Related papers  (2023-11-07T08:05:09Z)
• Heterogenous Memory Augmented Neural Networks [84.29338268789684]
  We introduce a novel heterogeneous memory augmentation approach for neural networks. By introducing learnable memory tokens with attention mechanism, we can effectively boost performance without huge computational overhead. We show our approach on various image and graph-based tasks under both in-distribution (ID) and out-of-distribution (OOD) conditions.
  arXiv  Detail & Related papers  (2023-10-17T01:05:28Z)
• Motor Imagery Classification based on CNN-GRU Network with Spatio-Temporal Feature Representation [22.488536453952964]
  Recently various deep neural networks have been applied to electroencephalogram (EEG) signal. EEG is a brain signal that can be acquired in a non-invasive way and has a high temporal resolution. As the EEG signal has a high dimension of classification feature space, appropriate feature extraction methods are needed to improve performance.
  arXiv  Detail & Related papers  (2021-07-15T01:05:38Z)
• Deepfake Detection Scheme Based on Vision Transformer and Distillation [4.716110829725784]
  We propose a Vision Transformer model with distillation methodology for detecting fake videos. We verify that the proposed scheme with patch embedding as input outperforms the state-of-the-art using the combined CNN features.
  arXiv  Detail & Related papers  (2021-04-03T09:13:05Z)
• Using CNNs to Identify the Origin of Finger Vein Image [6.954694117813895]
  We study the finger vein (FV) sensor model identification task using a deep learning approach. We employ five prominent CNN architectures covering a wide range of CNN family models. An excellent sensor identification AUC-ROC score of 1.0 for patches of uncropped samples and 0.9997 for ROI samples have been achieved.
  arXiv  Detail & Related papers  (2021-03-02T10:43:52Z)
• Adversarially robust deepfake media detection using fused convolutional neural network predictions [79.00202519223662]
  Current deepfake detection systems struggle against unseen data. We employ three different deep Convolutional Neural Network (CNN) models to classify fake and real images extracted from videos. The proposed technique outperforms state-of-the-art models with 96.5% accuracy.
  arXiv  Detail & Related papers  (2021-02-11T11:28:00Z)
• Perceptron Synthesis Network: Rethinking the Action Scale Variances in Videos [48.57686258913474]
  Video action recognition has been partially addressed by the CNNs stacking of fixed-size 3D kernels. We propose to learn the optimal-scale kernels from the data. An textitaction perceptron synthesizer is proposed to generate the kernels from a bag of fixed-size kernels.
  arXiv  Detail & Related papers  (2020-07-22T14:22:29Z)
• Deepfakes Detection with Automatic Face Weighting [21.723416806728668]
  We introduce a method based on convolutional neural networks (CNNs) and recurrent neural networks (RNNs) that extracts visual and temporal features from faces present in videos to accurately detect manipulations. The method is evaluated with the Deepfake Detection Challenge dataset, providing competitive results compared to other techniques.
  arXiv  Detail & Related papers  (2020-04-25T00:47:42Z)
• Cascaded Deep Video Deblurring Using Temporal Sharpness Prior [88.98348546566675]
  The proposed algorithm mainly consists of optical flow estimation from intermediate latent frames and latent frame restoration steps. It first develops a deep CNN model to estimate optical flow from intermediate latent frames and then restores the latent frames based on the estimated optical flow. We show that exploring the domain knowledge of video deblurring is able to make the deep CNN model more compact and efficient.
  arXiv  Detail & Related papers  (2020-04-06T09:13:49Z)
• Model Fusion via Optimal Transport [64.13185244219353]
  We present a layer-wise model fusion algorithm for neural networks. We show that this can successfully yield "one-shot" knowledge transfer between neural networks trained on heterogeneous non-i.i.d. data.
  arXiv  Detail & Related papers  (2019-10-12T22:07:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.