Learning to mask: Towards generalized face forgery detection
- URL: http://arxiv.org/abs/2212.14309v1
- Date: Thu, 29 Dec 2022 13:55:28 GMT
- Title: Learning to mask: Towards generalized face forgery detection
- Authors: Jianwei Fei, Yunshu Dai, Huaming Wang, Zhihua Xia
- Abstract summary: Generalizability to unseen forgery types is crucial for face forgery detectors.
Our goal is to reduce the features that are easy to learn in the training phase, so as to reduce the risk of overfitting on specific forgery types.
A deep feature mixup strategy is also proposed to synthesize forgeries in the feature domain.
- Score: 3.7755650136637304
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Generalizability to unseen forgery types is crucial for face forgery
detectors. Recent works have made significant progress in terms of
generalization by synthetic forgery data augmentation. In this work, we explore
another path for improving the generalization. Our goal is to reduce the
features that are easy to learn in the training phase, so as to reduce the risk
of overfitting on specific forgery types. Specifically, in our method, a
teacher network takes as input the face images and generates an attention map
of the deep features by a diverse multihead attention ViT. The attention map is
used to guide a student network to focus on the low-attended features by
reducing the highly-attended deep features. A deep feature mixup strategy is
also proposed to synthesize forgeries in the feature domain. Experiments
demonstrate that, without data augmentation, our method is able to achieve
promising performances on unseen forgeries and highly compressed data.
Related papers
- Appearance Debiased Gaze Estimation via Stochastic Subject-Wise
Adversarial Learning [33.55397868171977]
Appearance-based gaze estimation has been attracting attention in computer vision, and remarkable improvements have been achieved using various deep learning techniques.
We propose a novel framework: subject-wise gaZE learning (SAZE), which trains a network to generalize the appearance of subjects.
Our experimental results verify the robustness of the method in that it yields state-of-the-art performance, achieving 3.89 and 4.42 on the MPIIGaze and EyeDiap datasets, respectively.
arXiv Detail & Related papers (2024-01-25T00:23:21Z) - Attention Consistency Refined Masked Frequency Forgery Representation
for Generalizing Face Forgery Detection [96.539862328788]
Existing forgery detection methods suffer from unsatisfactory generalization ability to determine the authenticity in the unseen domain.
We propose a novel Attention Consistency Refined masked frequency forgery representation model toward generalizing face forgery detection algorithm (ACMF)
Experiment results on several public face forgery datasets demonstrate the superior performance of the proposed method compared with the state-of-the-art methods.
arXiv Detail & Related papers (2023-07-21T08:58:49Z) - Beyond the Prior Forgery Knowledge: Mining Critical Clues for General
Face Forgery Detection [61.74632676703288]
We propose a novel Critical Forgery Mining framework, which can be flexibly assembled with various backbones to boost generalization and performance.
Specifically, we first build a fine-grained triplet and suppress specific forgery traces through prior knowledge-agnostic data augmentation.
We then propose a fine-grained relation learning prototype to mine critical information in forgeries through instance and local similarity-aware losses.
arXiv Detail & Related papers (2023-04-24T23:02:27Z) - FedForgery: Generalized Face Forgery Detection with Residual Federated
Learning [87.746829550726]
Existing face forgery detection methods directly utilize the obtained public shared or centralized data for training.
The paper proposes a novel generalized residual Federated learning for face Forgery detection (FedForgery)
Experiments conducted on publicly available face forgery detection datasets prove the superior performance of the proposed FedForgery.
arXiv Detail & Related papers (2022-10-18T03:32:18Z) - MetAug: Contrastive Learning via Meta Feature Augmentation [28.708395209321846]
We argue that contrastive learning heavily relies on informative features, or "hard" (positive or negative) features.
The key challenge toward exploring such features is that the source multi-view data is generated by applying random data augmentations.
We propose to directly augment the features in latent space, thereby learning discriminative representations without a large amount of input data.
arXiv Detail & Related papers (2022-03-10T02:35:39Z) - Self-supervised Transformer for Deepfake Detection [112.81127845409002]
Deepfake techniques in real-world scenarios require stronger generalization abilities of face forgery detectors.
Inspired by transfer learning, neural networks pre-trained on other large-scale face-related tasks may provide useful features for deepfake detection.
In this paper, we propose a self-supervised transformer based audio-visual contrastive learning method.
arXiv Detail & Related papers (2022-03-02T17:44:40Z) - Towards Generalizable and Robust Face Manipulation Detection via
Bag-of-local-feature [55.47546606878931]
We propose a novel method for face manipulation detection, which can improve the generalization ability and robustness by bag-of-local-feature.
Specifically, we extend Transformers using bag-of-feature approach to encode inter-patch relationships, allowing it to learn local forgery features without any explicit supervision.
arXiv Detail & Related papers (2021-03-14T12:50:48Z) - Multi-attentional Deepfake Detection [79.80308897734491]
Face forgery by deepfake is widely spread over the internet and has raised severe societal concerns.
We propose a new multi-attentional deepfake detection network. Specifically, it consists of three key components: 1) multiple spatial attention heads to make the network attend to different local parts; 2) textural feature enhancement block to zoom in the subtle artifacts in shallow features; 3) aggregate the low-level textural feature and high-level semantic features guided by the attention maps.
arXiv Detail & Related papers (2021-03-03T13:56:14Z) - Hybrid-Attention Guided Network with Multiple Resolution Features for
Person Re-Identification [30.285126447140254]
We present a novel person re-ID model that fuses high- and low-level embeddings to reduce the information loss caused in learning high-level features.
We also introduce the spatial and channel attention mechanisms in our model, which aims to mine more discriminative features related to the target.
arXiv Detail & Related papers (2020-09-16T08:12:42Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.