Video Transformer for Deepfake Detection with Incremental Learning

• URL: http://arxiv.org/abs/2108.05307v1
• Date: Wed, 11 Aug 2021 16:22:56 GMT
• Title: Video Transformer for Deepfake Detection with Incremental Learning
• Authors: Sohail A. Khan and Hang Dai
• Abstract summary: Face forgery by deepfake is widely spread over the internet and this raises severe societal concerns. We propose a novel video transformer with incremental learning for detecting deepfake videos.
• Score: 11.586926513803077
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Face forgery by deepfake is widely spread over the internet and this raises severe societal concerns. In this paper, we propose a novel video transformer with incremental learning for detecting deepfake videos. To better align the input face images, we use a 3D face reconstruction method to generate UV texture from a single input face image. The aligned face image can also provide pose, eyes blink and mouth movement information that cannot be perceived in the UV texture image, so we use both face images and their UV texture maps to extract the image features. We present an incremental learning strategy to fine-tune the proposed model on a smaller amount of data and achieve better deepfake detection performance. The comprehensive experiments on various public deepfake datasets demonstrate that the proposed video transformer model with incremental learning achieves state-of-the-art performance in the deepfake video detection task with enhanced feature learning from the sequenced data.

Related papers

• Deepfake detection in videos with multiple faces using geometric-fakeness features [79.16635054977068]
  Deepfakes of victims or public figures can be used by fraudsters for blackmailing, extorsion and financial fraud. In our research we propose to use geometric-fakeness features (GFF) that characterize a dynamic degree of a face presence in a video. We employ our approach to analyze videos with multiple faces that are simultaneously present in a video.
  arXiv  Detail & Related papers  (2024-10-10T13:10:34Z)
• UV Gaussians: Joint Learning of Mesh Deformation and Gaussian Textures for Human Avatar Modeling [71.87807614875497]
  We propose UV Gaussians, which models the 3D human body by jointly learning mesh deformations and 2D UV-space Gaussian textures. We collect and process a new dataset of human motion, which includes multi-view images, scanned models, parametric model registration, and corresponding texture maps. Experimental results demonstrate that our method achieves state-of-the-art synthesis of novel view and novel pose.
  arXiv  Detail & Related papers  (2024-03-18T09:03:56Z)
• AntifakePrompt: Prompt-Tuned Vision-Language Models are Fake Image Detectors [24.78672820633581]
  Deep generative models can create remarkably fake images while raising concerns about misinformation and copyright infringement. Deepfake detection technique is developed to distinguish between real and fake images. We propose a novel approach called AntifakePrompt, using Vision-Language Models and prompt tuning techniques.
  arXiv  Detail & Related papers  (2023-10-26T14:23:45Z)
• Deepfake Video Detection Using Generative Convolutional Vision Transformer [3.8297637120486496]
  We propose a Generative Convolutional Vision Transformer (GenConViT) for deepfake video detection. Our model combines ConvNeXt and Swin Transformer models for feature extraction. By learning from the visual artifacts and latent data distribution, GenConViT achieves improved performance in detecting a wide range of deepfake videos.
  arXiv  Detail & Related papers  (2023-07-13T19:27:40Z)
• Undercover Deepfakes: Detecting Fake Segments in Videos [1.2609216345578933]
  deepfake generation is a new paradigm of deepfakes which are mostly real videos altered slightly to distort the truth. In this paper, we present a deepfake detection method that can address this issue by performing deepfake prediction at the frame and video levels. In particular, the paradigm we address will form a powerful tool for the moderation of deepfakes, where human oversight can be better targeted to the parts of videos suspected of being deepfakes.
  arXiv  Detail & Related papers  (2023-05-11T04:43:10Z)
• Deep Convolutional Pooling Transformer for Deepfake Detection [54.10864860009834]
  We propose a deep convolutional Transformer to incorporate decisive image features both locally and globally. Specifically, we apply convolutional pooling and re-attention to enrich the extracted features and enhance efficacy. The proposed solution consistently outperforms several state-of-the-art baselines on both within- and cross-dataset experiments.
  arXiv  Detail & Related papers  (2022-09-12T15:05:41Z)
• Copy Motion From One to Another: Fake Motion Video Generation [53.676020148034034]
  A compelling application of artificial intelligence is to generate a video of a target person performing arbitrary desired motion. Current methods typically employ GANs with a L2 loss to assess the authenticity of the generated videos. We propose a theoretically motivated Gromov-Wasserstein loss that facilitates learning the mapping from a pose to a foreground image. Our method is able to generate realistic target person videos, faithfully copying complex motions from a source person.
  arXiv  Detail & Related papers  (2022-05-03T08:45:22Z)
• DA-FDFtNet: Dual Attention Fake Detection Fine-tuning Network to Detect Various AI-Generated Fake Images [21.030153777110026]
  It has been much easier to create fake images such as "Deepfakes" Recent research has introduced few-shot learning, which uses a small amount of training data to produce fake images and videos more effectively. In this work, we propose Dual Attention Fine-tuning Network (DA-tNet) to detect the manipulated fake face images.
  arXiv  Detail & Related papers  (2021-12-22T16:25:24Z)
• M2TR: Multi-modal Multi-scale Transformers for Deepfake Detection [74.19291916812921]
  forged images generated by Deepfake techniques pose a serious threat to the trustworthiness of digital information. In this paper, we aim to capture the subtle manipulation artifacts at different scales for Deepfake detection. We introduce a high-quality Deepfake dataset, SR-DF, which consists of 4,000 DeepFake videos generated by state-of-the-art face swapping and facial reenactment methods.
  arXiv  Detail & Related papers  (2021-04-20T05:43:44Z)
• Deepfake Video Detection Using Convolutional Vision Transformer [0.0]
  Deep learning techniques can generate and synthesis hyper-realistic videos known as Deepfakes. Deepfakes pose a looming threat to everyone if used for harmful purposes such as identity theft, phishing, and scam. We propose a Convolutional Vision Transformer for the detection of Deepfakes.
  arXiv  Detail & Related papers  (2021-02-22T15:56:05Z)
• Sharp Multiple Instance Learning for DeepFake Video Detection [54.12548421282696]
  We introduce a new problem of partial face attack in DeepFake video, where only video-level labels are provided but not all the faces in the fake videos are manipulated. A sharp MIL (S-MIL) is proposed which builds direct mapping from instance embeddings to bag prediction. Experiments on FFPMS and widely used DFDC dataset verify that S-MIL is superior to other counterparts for partially attacked DeepFake video detection.
  arXiv  Detail & Related papers  (2020-08-11T08:52:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.