Multi-attentional Deepfake Detection

• URL: http://arxiv.org/abs/2103.02406v2
• Date: Fri, 5 Mar 2021 02:09:39 GMT
• Title: Multi-attentional Deepfake Detection
• Authors: Hanqing Zhao, Wenbo Zhou, Dongdong Chen, Tianyi Wei, Weiming Zhang, Nenghai Yu
• Abstract summary: Face forgery by deepfake is widely spread over the internet and has raised severe societal concerns. We propose a new multi-attentional deepfake detection network. Specifically, it consists of three key components: 1) multiple spatial attention heads to make the network attend to different local parts; 2) textural feature enhancement block to zoom in the subtle artifacts in shallow features; 3) aggregate the low-level textural feature and high-level semantic features guided by the attention maps.
• Score: 79.80308897734491
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Face forgery by deepfake is widely spread over the internet and has raised severe societal concerns. Recently, how to detect such forgery contents has become a hot research topic and many deepfake detection methods have been proposed. Most of them model deepfake detection as a vanilla binary classification problem, i.e, first use a backbone network to extract a global feature and then feed it into a binary classifier (real/fake). But since the difference between the real and fake images in this task is often subtle and local, we argue this vanilla solution is not optimal. In this paper, we instead formulate deepfake detection as a fine-grained classification problem and propose a new multi-attentional deepfake detection network. Specifically, it consists of three key components: 1) multiple spatial attention heads to make the network attend to different local parts; 2) textural feature enhancement block to zoom in the subtle artifacts in shallow features; 3) aggregate the low-level textural feature and high-level semantic features guided by the attention maps. Moreover, to address the learning difficulty of this network, we further introduce a new regional independence loss and an attention guided data augmentation strategy. Through extensive experiments on different datasets, we demonstrate the superiority of our method over the vanilla binary classifier counterparts, and achieve state-of-the-art performance.

Related papers

• DF40: Toward Next-Generation Deepfake Detection [62.073997142001424]
  existing works identify top-notch detection algorithms and models by adhering to the common practice: training detectors on one specific dataset and testing them on other prevalent deepfake datasets. But can these stand-out "winners" be truly applied to tackle the myriad of realistic and diverse deepfakes lurking in the real world? We construct a highly diverse deepfake detection dataset called DF40, which comprises 40 distinct deepfake techniques.
  arXiv  Detail & Related papers  (2024-06-19T12:35:02Z)
• Exposing the Deception: Uncovering More Forgery Clues for Deepfake Detection [36.92399832886853]
  Current deepfake detection approaches may easily fall into the trap of overfitting, focusing only on forgery clues within one or a few local regions. We present a novel framework to capture broader forgery clues by extracting multiple non-overlapping local representations and fusing them into a global semantic-rich feature. Our method achieves state-of-the-art performance on five benchmark datasets.
  arXiv  Detail & Related papers  (2024-03-04T07:28:23Z)
• Facial Forgery-based Deepfake Detection using Fine-Grained Features [7.378937711027777]
  Facial forgery by deepfakes has caused major security risks and raised severe societal concerns. We formulate deepfake detection as a fine-grained classification problem and propose a new fine-grained solution to it. Our method is based on learning subtle and generalizable features by effectively suppressing background noise and learning discriminative features at various scales for deepfake detection.
  arXiv  Detail & Related papers  (2023-10-10T21:30:05Z)
• Deepfake Detection via Joint Unsupervised Reconstruction and Supervised Classification [25.84902508816679]
  We introduce a novel approach for deepfake detection, which considers the reconstruction and classification tasks simultaneously. This method shares the information learned by one task with the other, which focuses on a different aspect other existing works rarely consider. Our method achieves state-of-the-art performance on three commonly-used datasets.
  arXiv  Detail & Related papers  (2022-11-24T05:44:26Z)
• Aggregating Layers for Deepfake Detection [20.191456827448736]
  We consider the case where the network is trained on one Deepfake algorithm, and tested on Deepfakes generated by another algorithm. Our algorithm aggregates features extracted across all layers of one backbone network to detect a fake. We evaluate our approach on two domains of interest - Deepfake detection and Synthetic image detection, and find that we achieve SOTA results.
  arXiv  Detail & Related papers  (2022-10-11T14:29:47Z)
• Deep Convolutional Pooling Transformer for Deepfake Detection [54.10864860009834]
  We propose a deep convolutional Transformer to incorporate decisive image features both locally and globally. Specifically, we apply convolutional pooling and re-attention to enrich the extracted features and enhance efficacy. The proposed solution consistently outperforms several state-of-the-art baselines on both within- and cross-dataset experiments.
  arXiv  Detail & Related papers  (2022-09-12T15:05:41Z)
• Detection of Deepfake Videos Using Long Distance Attention [73.6659488380372]
  Most existing detection methods treat the problem as a vanilla binary classification problem. In this paper, the problem is treated as a special fine-grained classification problem since the differences between fake and real faces are very subtle. A spatial-temporal model is proposed which has two components for capturing spatial and temporal forgery traces in global perspective.
  arXiv  Detail & Related papers  (2021-06-24T08:33:32Z)
• M2TR: Multi-modal Multi-scale Transformers for Deepfake Detection [74.19291916812921]
  forged images generated by Deepfake techniques pose a serious threat to the trustworthiness of digital information. In this paper, we aim to capture the subtle manipulation artifacts at different scales for Deepfake detection. We introduce a high-quality Deepfake dataset, SR-DF, which consists of 4,000 DeepFake videos generated by state-of-the-art face swapping and facial reenactment methods.
  arXiv  Detail & Related papers  (2021-04-20T05:43:44Z)
• Generalizing Face Forgery Detection with High-frequency Features [63.33397573649408]
  Current CNN-based detectors tend to overfit to method-specific color textures and thus fail to generalize. We propose to utilize the high-frequency noises for face forgery detection. The first is the multi-scale high-frequency feature extraction module that extracts high-frequency noises at multiple scales. The second is the residual-guided spatial attention module that guides the low-level RGB feature extractor to concentrate more on forgery traces from a new perspective.
  arXiv  Detail & Related papers  (2021-03-23T08:19:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.