Beyond the Prior Forgery Knowledge: Mining Critical Clues for General Face Forgery Detection

• URL: http://arxiv.org/abs/2304.12489v1
• Date: Mon, 24 Apr 2023 23:02:27 GMT
• Title: Beyond the Prior Forgery Knowledge: Mining Critical Clues for General Face Forgery Detection
• Authors: Anwei Luo, Chenqi Kong, Jiwu Huang, Yongjian Hu, Xiangui Kang and Alex C. Kot
• Abstract summary: We propose a novel Critical Forgery Mining framework, which can be flexibly assembled with various backbones to boost generalization and performance. Specifically, we first build a fine-grained triplet and suppress specific forgery traces through prior knowledge-agnostic data augmentation. We then propose a fine-grained relation learning prototype to mine critical information in forgeries through instance and local similarity-aware losses.
• Score: 61.74632676703288
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Face forgery detection is essential in combating malicious digital face attacks. Previous methods mainly rely on prior expert knowledge to capture specific forgery clues, such as noise patterns, blending boundaries, and frequency artifacts. However, these methods tend to get trapped in local optima, resulting in limited robustness and generalization capability. To address these issues, we propose a novel Critical Forgery Mining (CFM) framework, which can be flexibly assembled with various backbones to boost their generalization and robustness performance. Specifically, we first build a fine-grained triplet and suppress specific forgery traces through prior knowledge-agnostic data augmentation. Subsequently, we propose a fine-grained relation learning prototype to mine critical information in forgeries through instance and local similarity-aware losses. Moreover, we design a novel progressive learning controller to guide the model to focus on principal feature components, enabling it to learn critical forgery features in a coarse-to-fine manner. The proposed method achieves state-of-the-art forgery detection performance under various challenging evaluation settings.

Related papers

• Open-Set Deepfake Detection: A Parameter-Efficient Adaptation Method with Forgery Style Mixture [58.60915132222421]
  We introduce an approach that is both general and parameter-efficient for face forgery detection. We design a forgery-style mixture formulation that augments the diversity of forgery source domains. We show that the designed model achieves state-of-the-art generalizability with significantly reduced trainable parameters.
  arXiv  Detail & Related papers  (2024-08-23T01:53:36Z)
• UniForensics: Face Forgery Detection via General Facial Representation [60.5421627990707]
  High-level semantic features are less susceptible to perturbations and not limited to forgery-specific artifacts, thus having stronger generalization. We introduce UniForensics, a novel deepfake detection framework that leverages a transformer-based video network, with a meta-functional face classification for enriched facial representation.
  arXiv  Detail & Related papers  (2024-07-26T20:51:54Z)
• Towards General Visual-Linguistic Face Forgery Detection [95.73987327101143]
  Deepfakes are realistic face manipulations that can pose serious threats to security, privacy, and trust. Existing methods mostly treat this task as binary classification, which uses digital labels or mask signals to train the detection model. We propose a novel paradigm named Visual-Linguistic Face Forgery Detection(VLFFD), which uses fine-grained sentence-level prompts as the annotation.
  arXiv  Detail & Related papers  (2023-07-31T10:22:33Z)
• Attention Consistency Refined Masked Frequency Forgery Representation for Generalizing Face Forgery Detection [96.539862328788]
  Existing forgery detection methods suffer from unsatisfactory generalization ability to determine the authenticity in the unseen domain. We propose a novel Attention Consistency Refined masked frequency forgery representation model toward generalizing face forgery detection algorithm (ACMF) Experiment results on several public face forgery datasets demonstrate the superior performance of the proposed method compared with the state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-07-21T08:58:49Z)
• UCF: Uncovering Common Features for Generalizable Deepfake Detection [44.12640679000489]
  Deepfake detection remains a challenging task due to the difficulty of generalizing to new types of forgeries. This paper presents a novel approach to address the two types of overfitting issues by uncovering common forgery features. Our framework can perform superior generalization than current state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-04-27T04:07:29Z)
• Learning to mask: Towards generalized face forgery detection [3.7755650136637304]
  Generalizability to unseen forgery types is crucial for face forgery detectors. Our goal is to reduce the features that are easy to learn in the training phase, so as to reduce the risk of overfitting on specific forgery types. A deep feature mixup strategy is also proposed to synthesize forgeries in the feature domain.
  arXiv  Detail & Related papers  (2022-12-29T13:55:28Z)
• Dual Contrastive Learning for General Face Forgery Detection [64.41970626226221]
  We propose a novel face forgery detection framework, named Dual Contrastive Learning (DCL), which constructs positive and negative paired data. To explore the essential discrepancies, Intra-Instance Contrastive Learning (Intra-ICL) is introduced to focus on the local content inconsistencies prevalent in the forged faces.
  arXiv  Detail & Related papers  (2021-12-27T05:44:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.