Learning to Recognize Patch-Wise Consistency for Deepfake Detection
- URL: http://arxiv.org/abs/2012.09311v1
- Date: Wed, 16 Dec 2020 23:06:56 GMT
- Title: Learning to Recognize Patch-Wise Consistency for Deepfake Detection
- Authors: Tianchen Zhao, Xiang Xu, Mingze Xu, Hui Ding, Yuanjun Xiong, Wei Xia
- Abstract summary: We propose a representation learning approach for this task, called patch-wise consistency learning (PCL)
PCL learns by measuring the consistency of image source features, resulting to representation with good interpretability and robustness to multiple forgery methods.
We evaluate our approach on seven popular Deepfake detection datasets.
- Score: 39.186451993950044
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: We propose to detect Deepfake generated by face manipulation based on one of
their fundamental features: images are blended by patches from multiple
sources, carrying distinct and persistent source features. In particular, we
propose a novel representation learning approach for this task, called
patch-wise consistency learning (PCL). It learns by measuring the consistency
of image source features, resulting to representation with good
interpretability and robustness to multiple forgery methods. We develop an
inconsistency image generator (I2G) to generate training data for PCL and boost
its robustness. We evaluate our approach on seven popular Deepfake detection
datasets. Our model achieves superior detection accuracy and generalizes well
to unseen generation methods. On average, our model outperforms the
state-of-the-art in terms of AUC by 2% and 8% in the in- and cross-dataset
evaluation, respectively.
Related papers
- Understanding and Improving Training-Free AI-Generated Image Detections with Vision Foundation Models [68.90917438865078]
Deepfake techniques for facial synthesis and editing pose serious risks for generative models.
In this paper, we investigate how detection performance varies across model backbones, types, and datasets.
We introduce Contrastive Blur, which enhances performance on facial images, and MINDER, which addresses noise type bias, balancing performance across domains.
arXiv Detail & Related papers (2024-11-28T13:04:45Z) - Standing on the Shoulders of Giants: Reprogramming Visual-Language Model for General Deepfake Detection [16.21235742118949]
We propose a novel approach that repurposes a well-trained Vision-Language Models (VLMs) for general deepfake detection.
Motivated by the model reprogramming paradigm that manipulates the model prediction via input perturbations, our method can reprogram a pre-trained VLM model.
Experiments on several popular benchmark datasets demonstrate that the cross-dataset and cross-manipulation performances of deepfake detection can be significantly and consistently improved.
arXiv Detail & Related papers (2024-09-04T12:46:30Z) - UniForensics: Face Forgery Detection via General Facial Representation [60.5421627990707]
High-level semantic features are less susceptible to perturbations and not limited to forgery-specific artifacts, thus having stronger generalization.
We introduce UniForensics, a novel deepfake detection framework that leverages a transformer-based video network, with a meta-functional face classification for enriched facial representation.
arXiv Detail & Related papers (2024-07-26T20:51:54Z) - Towards More General Video-based Deepfake Detection through Facial Feature Guided Adaptation for Foundation Model [15.61920157541529]
We propose a novel Deepfake detection approach by adapting the Foundation Models with rich information encoded inside.
Inspired by the recent advances of parameter efficient fine-tuning, we propose a novel side-network-based decoder.
Our approach exhibits superior effectiveness in identifying unseen Deepfake samples, achieving notable performance improvement.
arXiv Detail & Related papers (2024-04-08T14:58:52Z) - DFIL: Deepfake Incremental Learning by Exploiting Domain-invariant
Forgery Clues [32.045504965382015]
Current deepfake detection models can generally recognize forgery images by training on a large dataset.
The accuracy of detection models degrades significantly on images generated by new deepfake methods due to the difference in data distribution.
We present a novel incremental learning framework that improves the generalization of deepfake detection models.
arXiv Detail & Related papers (2023-09-18T07:02:26Z) - Deep Convolutional Pooling Transformer for Deepfake Detection [54.10864860009834]
We propose a deep convolutional Transformer to incorporate decisive image features both locally and globally.
Specifically, we apply convolutional pooling and re-attention to enrich the extracted features and enhance efficacy.
The proposed solution consistently outperforms several state-of-the-art baselines on both within- and cross-dataset experiments.
arXiv Detail & Related papers (2022-09-12T15:05:41Z) - Detecting Deepfakes with Self-Blended Images [37.374772758057844]
We present novel synthetic training data called self-blended images ( SBIs) to detect deepfakes.
SBIs are generated by blending pseudo source and target images from single pristine images.
We compare our approach with state-of-the-art methods on FF++, CDF, DFD, DFDC, DFDCP, and FFIW datasets.
arXiv Detail & Related papers (2022-04-18T15:44:35Z) - M2TR: Multi-modal Multi-scale Transformers for Deepfake Detection [74.19291916812921]
forged images generated by Deepfake techniques pose a serious threat to the trustworthiness of digital information.
In this paper, we aim to capture the subtle manipulation artifacts at different scales for Deepfake detection.
We introduce a high-quality Deepfake dataset, SR-DF, which consists of 4,000 DeepFake videos generated by state-of-the-art face swapping and facial reenactment methods.
arXiv Detail & Related papers (2021-04-20T05:43:44Z) - CutPaste: Self-Supervised Learning for Anomaly Detection and
Localization [59.719925639875036]
We propose a framework for building anomaly detectors using normal training data only.
We first learn self-supervised deep representations and then build a generative one-class classifier on learned representations.
Our empirical study on MVTec anomaly detection dataset demonstrates the proposed algorithm is general to be able to detect various types of real-world defects.
arXiv Detail & Related papers (2021-04-08T19:04:55Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.