Beyond the Prior Forgery Knowledge: Mining Critical Clues for General
Face Forgery Detection
- URL: http://arxiv.org/abs/2304.12489v1
- Date: Mon, 24 Apr 2023 23:02:27 GMT
- Title: Beyond the Prior Forgery Knowledge: Mining Critical Clues for General
Face Forgery Detection
- Authors: Anwei Luo, Chenqi Kong, Jiwu Huang, Yongjian Hu, Xiangui Kang and Alex
C. Kot
- Abstract summary: We propose a novel Critical Forgery Mining framework, which can be flexibly assembled with various backbones to boost generalization and performance.
Specifically, we first build a fine-grained triplet and suppress specific forgery traces through prior knowledge-agnostic data augmentation.
We then propose a fine-grained relation learning prototype to mine critical information in forgeries through instance and local similarity-aware losses.
- Score: 61.74632676703288
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Face forgery detection is essential in combating malicious digital face
attacks. Previous methods mainly rely on prior expert knowledge to capture
specific forgery clues, such as noise patterns, blending boundaries, and
frequency artifacts. However, these methods tend to get trapped in local
optima, resulting in limited robustness and generalization capability. To
address these issues, we propose a novel Critical Forgery Mining (CFM)
framework, which can be flexibly assembled with various backbones to boost
their generalization and robustness performance. Specifically, we first build a
fine-grained triplet and suppress specific forgery traces through prior
knowledge-agnostic data augmentation. Subsequently, we propose a fine-grained
relation learning prototype to mine critical information in forgeries through
instance and local similarity-aware losses. Moreover, we design a novel
progressive learning controller to guide the model to focus on principal
feature components, enabling it to learn critical forgery features in a
coarse-to-fine manner. The proposed method achieves state-of-the-art forgery
detection performance under various challenging evaluation settings.
Related papers
- Transcending Forgery Specificity with Latent Space Augmentation for Generalizable Deepfake Detection [57.646582245834324]
We propose a simple yet effective deepfake detector called LSDA.
It is based on a idea: representations with a wider variety of forgeries should be able to learn a more generalizable decision boundary.
We show that our proposed method is surprisingly effective and transcends state-of-the-art detectors across several widely used benchmarks.
arXiv Detail & Related papers (2023-11-19T09:41:10Z) - Towards General Visual-Linguistic Face Forgery Detection [95.73987327101143]
Deepfakes are realistic face manipulations that can pose serious threats to security, privacy, and trust.
Existing methods mostly treat this task as binary classification, which uses digital labels or mask signals to train the detection model.
We propose a novel paradigm named Visual-Linguistic Face Forgery Detection(VLFFD), which uses fine-grained sentence-level prompts as the annotation.
arXiv Detail & Related papers (2023-07-31T10:22:33Z) - Attention Consistency Refined Masked Frequency Forgery Representation
for Generalizing Face Forgery Detection [96.539862328788]
Existing forgery detection methods suffer from unsatisfactory generalization ability to determine the authenticity in the unseen domain.
We propose a novel Attention Consistency Refined masked frequency forgery representation model toward generalizing face forgery detection algorithm (ACMF)
Experiment results on several public face forgery datasets demonstrate the superior performance of the proposed method compared with the state-of-the-art methods.
arXiv Detail & Related papers (2023-07-21T08:58:49Z) - UCF: Uncovering Common Features for Generalizable Deepfake Detection [44.12640679000489]
Deepfake detection remains a challenging task due to the difficulty of generalizing to new types of forgeries.
This paper presents a novel approach to address the two types of overfitting issues by uncovering common forgery features.
Our framework can perform superior generalization than current state-of-the-art methods.
arXiv Detail & Related papers (2023-04-27T04:07:29Z) - Learning to mask: Towards generalized face forgery detection [3.7755650136637304]
Generalizability to unseen forgery types is crucial for face forgery detectors.
Our goal is to reduce the features that are easy to learn in the training phase, so as to reduce the risk of overfitting on specific forgery types.
A deep feature mixup strategy is also proposed to synthesize forgeries in the feature domain.
arXiv Detail & Related papers (2022-12-29T13:55:28Z) - FedForgery: Generalized Face Forgery Detection with Residual Federated
Learning [87.746829550726]
Existing face forgery detection methods directly utilize the obtained public shared or centralized data for training.
The paper proposes a novel generalized residual Federated learning for face Forgery detection (FedForgery)
Experiments conducted on publicly available face forgery detection datasets prove the superior performance of the proposed FedForgery.
arXiv Detail & Related papers (2022-10-18T03:32:18Z) - Dual Contrastive Learning for General Face Forgery Detection [64.41970626226221]
We propose a novel face forgery detection framework, named Dual Contrastive Learning (DCL), which constructs positive and negative paired data.
To explore the essential discrepancies, Intra-Instance Contrastive Learning (Intra-ICL) is introduced to focus on the local content inconsistencies prevalent in the forged faces.
arXiv Detail & Related papers (2021-12-27T05:44:40Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.