Learning to mask: Towards generalized face forgery detection

• URL: http://arxiv.org/abs/2212.14309v2
• Date: Mon, 18 Nov 2024 17:14:42 GMT
• Title: Learning to mask: Towards generalized face forgery detection
• Authors: Jianwei Fei, Yunshu Dai, Huaming Wang, Zhihua Xia,
• Abstract summary: Generalizability to unseen forgery types is crucial for face forgery detectors. Our goal is to reduce the features that are easy to learn in the training phase, so as to reduce the risk of overfitting on specific forgery types. A deep feature mixup strategy is also proposed to synthesize forgeries in the feature domain.
• Score: 10.155873909545198
• License:
• Abstract: Generalizability to unseen forgery types is crucial for face forgery detectors. Recent works have made significant progress in terms of generalization by synthetic forgery data augmentation. In this work, we explore another path for improving the generalization. Our goal is to reduce the features that are easy to learn in the training phase, so as to reduce the risk of overfitting on specific forgery types. Specifically, in our method, a teacher network takes as input the face images and generates an attention map of the deep features by a diverse multihead attention ViT. The attention map is used to guide a student network to focus on the low-attended features by reducing the highly-attended deep features. A deep feature mixup strategy is also proposed to synthesize forgeries in the feature domain. Experiments demonstrate that, without data augmentation, our method is able to achieve promising performances on unseen forgeries and highly compressed data.

Related papers

• Semantics-Oriented Multitask Learning for DeepFake Detection: A Joint Embedding Approach [77.65459419417533]
  We propose an automatic dataset expansion technique to support semantics-oriented DeepFake detection tasks. We also resort to joint embedding of face images and their corresponding labels for prediction. Our method improves the generalizability of DeepFake detection and renders some degree of model interpretation by providing human-understandable explanations.
  arXiv  Detail & Related papers  (2024-08-29T07:11:50Z)
• UniForensics: Face Forgery Detection via General Facial Representation [60.5421627990707]
  High-level semantic features are less susceptible to perturbations and not limited to forgery-specific artifacts, thus having stronger generalization. We introduce UniForensics, a novel deepfake detection framework that leverages a transformer-based video network, with a meta-functional face classification for enriched facial representation.
  arXiv  Detail & Related papers  (2024-07-26T20:51:54Z)
• Appearance Debiased Gaze Estimation via Stochastic Subject-Wise Adversarial Learning [33.55397868171977]
  Appearance-based gaze estimation has been attracting attention in computer vision, and remarkable improvements have been achieved using various deep learning techniques. We propose a novel framework: subject-wise gaZE learning (SAZE), which trains a network to generalize the appearance of subjects. Our experimental results verify the robustness of the method in that it yields state-of-the-art performance, achieving 3.89 and 4.42 on the MPIIGaze and EyeDiap datasets, respectively.
  arXiv  Detail & Related papers  (2024-01-25T00:23:21Z)
• Attention Consistency Refined Masked Frequency Forgery Representation for Generalizing Face Forgery Detection [96.539862328788]
  Existing forgery detection methods suffer from unsatisfactory generalization ability to determine the authenticity in the unseen domain. We propose a novel Attention Consistency Refined masked frequency forgery representation model toward generalizing face forgery detection algorithm (ACMF) Experiment results on several public face forgery datasets demonstrate the superior performance of the proposed method compared with the state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-07-21T08:58:49Z)
• Beyond the Prior Forgery Knowledge: Mining Critical Clues for General Face Forgery Detection [61.74632676703288]
  We propose a novel Critical Forgery Mining framework, which can be flexibly assembled with various backbones to boost generalization and performance. Specifically, we first build a fine-grained triplet and suppress specific forgery traces through prior knowledge-agnostic data augmentation. We then propose a fine-grained relation learning prototype to mine critical information in forgeries through instance and local similarity-aware losses.
  arXiv  Detail & Related papers  (2023-04-24T23:02:27Z)
• MetAug: Contrastive Learning via Meta Feature Augmentation [28.708395209321846]
  We argue that contrastive learning heavily relies on informative features, or "hard" (positive or negative) features. The key challenge toward exploring such features is that the source multi-view data is generated by applying random data augmentations. We propose to directly augment the features in latent space, thereby learning discriminative representations without a large amount of input data.
  arXiv  Detail & Related papers  (2022-03-10T02:35:39Z)
• Self-supervised Transformer for Deepfake Detection [112.81127845409002]
  Deepfake techniques in real-world scenarios require stronger generalization abilities of face forgery detectors. Inspired by transfer learning, neural networks pre-trained on other large-scale face-related tasks may provide useful features for deepfake detection. In this paper, we propose a self-supervised transformer based audio-visual contrastive learning method.
  arXiv  Detail & Related papers  (2022-03-02T17:44:40Z)
• Multi-attentional Deepfake Detection [79.80308897734491]
  Face forgery by deepfake is widely spread over the internet and has raised severe societal concerns. We propose a new multi-attentional deepfake detection network. Specifically, it consists of three key components: 1) multiple spatial attention heads to make the network attend to different local parts; 2) textural feature enhancement block to zoom in the subtle artifacts in shallow features; 3) aggregate the low-level textural feature and high-level semantic features guided by the attention maps.
  arXiv  Detail & Related papers  (2021-03-03T13:56:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.