Facial Forgery-based Deepfake Detection using Fine-Grained Features
- URL: http://arxiv.org/abs/2310.07028v1
- Date: Tue, 10 Oct 2023 21:30:05 GMT
- Title: Facial Forgery-based Deepfake Detection using Fine-Grained Features
- Authors: Aakash Varma Nadimpalli, Ajita Rattani
- Abstract summary: Facial forgery by deepfakes has caused major security risks and raised severe societal concerns.
We formulate deepfake detection as a fine-grained classification problem and propose a new fine-grained solution to it.
Our method is based on learning subtle and generalizable features by effectively suppressing background noise and learning discriminative features at various scales for deepfake detection.
- Score: 7.378937711027777
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Facial forgery by deepfakes has caused major security risks and raised severe
societal concerns. As a countermeasure, a number of deepfake detection methods
have been proposed. Most of them model deepfake detection as a binary
classification problem using a backbone convolutional neural network (CNN)
architecture pretrained for the task. These CNN-based methods have demonstrated
very high efficacy in deepfake detection with the Area under the Curve (AUC) as
high as $0.99$. However, the performance of these methods degrades
significantly when evaluated across datasets and deepfake manipulation
techniques. This draws our attention towards learning more subtle, local, and
discriminative features for deepfake detection. In this paper, we formulate
deepfake detection as a fine-grained classification problem and propose a new
fine-grained solution to it. Specifically, our method is based on learning
subtle and generalizable features by effectively suppressing background noise
and learning discriminative features at various scales for deepfake detection.
Through extensive experimental validation, we demonstrate the superiority of
our method over the published research in cross-dataset and cross-manipulation
generalization of deepfake detectors for the majority of the experimental
scenarios.
Related papers
- DF40: Toward Next-Generation Deepfake Detection [62.073997142001424]
Existing works identify top-notch detection algorithms and models by adhering to the common practice: training detectors on one specific dataset (e.g., FF++) and testing them on other prevalent deepfake datasets.
But can these stand-out "winners" be truly applied to tackle the myriad of realistic and diverse deepfakes lurking in the real world?
We construct a highly diverse and large-scale deepfake dataset called DF40, which comprises 40 distinct deepfake techniques.
We then conduct comprehensive evaluations using 4 standard evaluation protocols and 7 representative detectors, resulting in over 2,000 evaluations.
arXiv Detail & Related papers (2024-06-19T12:35:02Z) - Improving Cross-dataset Deepfake Detection with Deep Information
Decomposition [57.284370468207214]
Deepfake technology poses a significant threat to security and social trust.
Existing detection methods suffer from sharp performance degradation when faced with cross-dataset scenarios.
We propose a deep information decomposition (DID) framework in this paper.
arXiv Detail & Related papers (2023-09-30T12:30:25Z) - Aggregating Layers for Deepfake Detection [20.191456827448736]
We consider the case where the network is trained on one Deepfake algorithm, and tested on Deepfakes generated by another algorithm.
Our algorithm aggregates features extracted across all layers of one backbone network to detect a fake.
We evaluate our approach on two domains of interest - Deepfake detection and Synthetic image detection, and find that we achieve SOTA results.
arXiv Detail & Related papers (2022-10-11T14:29:47Z) - Deep Convolutional Pooling Transformer for Deepfake Detection [54.10864860009834]
We propose a deep convolutional Transformer to incorporate decisive image features both locally and globally.
Specifically, we apply convolutional pooling and re-attention to enrich the extracted features and enhance efficacy.
The proposed solution consistently outperforms several state-of-the-art baselines on both within- and cross-dataset experiments.
arXiv Detail & Related papers (2022-09-12T15:05:41Z) - Delving into Sequential Patches for Deepfake Detection [64.19468088546743]
Recent advances in face forgery techniques produce nearly untraceable deepfake videos, which could be leveraged with malicious intentions.
Previous studies has identified the importance of local low-level cues and temporal information in pursuit to generalize well across deepfake methods.
We propose the Local- & Temporal-aware Transformer-based Deepfake Detection framework, which adopts a local-to-global learning protocol.
arXiv Detail & Related papers (2022-07-06T16:46:30Z) - On Improving Cross-dataset Generalization of Deepfake Detectors [1.0152838128195467]
Facial manipulation by deep fake has caused major security risks and raised severe societal concerns.
We formulate deep fake detection as a hybrid combination of supervised and reinforcement learning (RL) to improve its cross-dataset generalization performance.
We demonstrate the superiority of our method over existing published research in cross-dataset generalization of deep fake detectors, thus obtaining state-of-the-art performance.
arXiv Detail & Related papers (2022-04-08T20:34:53Z) - An Experimental Evaluation on Deepfake Detection using Deep Face
Recognition [0.0]
Deep learning has led to the generation of very realistic fake content, also known as deepfakes.
Most of the current deepfake detection methods are deemed as a binary classification problem in distinguishing authentic images or videos from fake ones using two-class convolutional neural networks (CNNs)
This paper thoroughly evaluate the efficacy of deep face recognition in identifying deepfakes, using different loss functions and deepfake generation techniques.
arXiv Detail & Related papers (2021-10-04T18:02:56Z) - TAR: Generalized Forensic Framework to Detect Deepfakes using Weakly
Supervised Learning [17.40885531847159]
Deepfakes have become a critical social problem, and detecting them is of utmost importance.
In this work, we introduce a practical digital forensic tool to detect different types of deepfakes simultaneously.
We develop an autoencoder-based detection model with Residual blocks and sequentially perform transfer learning to detect different types of deepfakes simultaneously.
arXiv Detail & Related papers (2021-05-13T07:31:08Z) - Representative Forgery Mining for Fake Face Detection [52.896286647898386]
We propose an attention-based data augmentation framework to guide detector refine and enlarge its attention.
Our method tracks and occludes the Top-N sensitive facial regions, encouraging the detector to mine deeper into the regions ignored before for more representative forgery.
arXiv Detail & Related papers (2021-04-14T03:24:19Z) - Multi-attentional Deepfake Detection [79.80308897734491]
Face forgery by deepfake is widely spread over the internet and has raised severe societal concerns.
We propose a new multi-attentional deepfake detection network. Specifically, it consists of three key components: 1) multiple spatial attention heads to make the network attend to different local parts; 2) textural feature enhancement block to zoom in the subtle artifacts in shallow features; 3) aggregate the low-level textural feature and high-level semantic features guided by the attention maps.
arXiv Detail & Related papers (2021-03-03T13:56:14Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.