Improving Cross-dataset Deepfake Detection with Deep Information
Decomposition
- URL: http://arxiv.org/abs/2310.00359v1
- Date: Sat, 30 Sep 2023 12:30:25 GMT
- Title: Improving Cross-dataset Deepfake Detection with Deep Information
Decomposition
- Authors: Shanmin Yang, Shu Hu, Bin Zhu, Ying Fu, Siwei Lyu, Xi Wu, Xin Wang
- Abstract summary: Deepfake technology poses a significant threat to security and social trust.
Existing detection methods suffer from sharp performance degradation when faced with cross-dataset scenarios.
We propose a deep information decomposition (DID) framework in this paper.
- Score: 57.284370468207214
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Deepfake technology poses a significant threat to security and social trust.
Although existing detection methods have demonstrated high performance in
identifying forgeries within datasets using the same techniques for training
and testing, they suffer from sharp performance degradation when faced with
cross-dataset scenarios where unseen deepfake techniques are tested. To address
this challenge, we propose a deep information decomposition (DID) framework in
this paper. Unlike most existing deepfake detection methods, our framework
prioritizes high-level semantic features over visual artifacts. Specifically,
it decomposes facial features into deepfake-related and irrelevant information
and optimizes the deepfake information for real/fake discrimination to be
independent of other factors. Our approach improves the robustness of deepfake
detection against various irrelevant information changes and enhances the
generalization ability of the framework to detect unseen forgery methods.
Extensive experimental comparisons with existing state-of-the-art detection
methods validate the effectiveness and superiority of the DID framework on
cross-dataset deepfake detection.
Related papers
- Explicit Correlation Learning for Generalizable Cross-Modal Deepfake Detection [33.20064862916194]
This paper aims to learn potential cross-modal correlation to enhance deepfake detection towards various generation scenarios.
Our approach introduces a correlation distillation task, which models the inherent cross-modal correlation based on content information.
We present the Cross-Modal Deepfake dataset with four generation methods to evaluate the detection of diverse cross-modal deepfakes.
arXiv Detail & Related papers (2024-04-30T00:25:44Z) - DeepFidelity: Perceptual Forgery Fidelity Assessment for Deepfake
Detection [67.3143177137102]
Deepfake detection refers to detecting artificially generated or edited faces in images or videos.
We propose a novel Deepfake detection framework named DeepFidelity to adaptively distinguish real and fake faces.
arXiv Detail & Related papers (2023-12-07T07:19:45Z) - Facial Forgery-based Deepfake Detection using Fine-Grained Features [7.378937711027777]
Facial forgery by deepfakes has caused major security risks and raised severe societal concerns.
We formulate deepfake detection as a fine-grained classification problem and propose a new fine-grained solution to it.
Our method is based on learning subtle and generalizable features by effectively suppressing background noise and learning discriminative features at various scales for deepfake detection.
arXiv Detail & Related papers (2023-10-10T21:30:05Z) - Assessment Framework for Deepfake Detection in Real-world Situations [13.334500258498798]
Deep learning-based deepfake detection methods have exhibited remarkable performance.
The impact of various image and video processing operations and typical workflow distortions on detection accuracy has not been systematically measured.
A more reliable assessment framework is proposed to evaluate the performance of learning-based deepfake detectors in more realistic settings.
arXiv Detail & Related papers (2023-04-12T19:09:22Z) - Deep Convolutional Pooling Transformer for Deepfake Detection [54.10864860009834]
We propose a deep convolutional Transformer to incorporate decisive image features both locally and globally.
Specifically, we apply convolutional pooling and re-attention to enrich the extracted features and enhance efficacy.
The proposed solution consistently outperforms several state-of-the-art baselines on both within- and cross-dataset experiments.
arXiv Detail & Related papers (2022-09-12T15:05:41Z) - Delving into Sequential Patches for Deepfake Detection [64.19468088546743]
Recent advances in face forgery techniques produce nearly untraceable deepfake videos, which could be leveraged with malicious intentions.
Previous studies has identified the importance of local low-level cues and temporal information in pursuit to generalize well across deepfake methods.
We propose the Local- & Temporal-aware Transformer-based Deepfake Detection framework, which adopts a local-to-global learning protocol.
arXiv Detail & Related papers (2022-07-06T16:46:30Z) - A Continual Deepfake Detection Benchmark: Dataset, Methods, and
Essentials [97.69553832500547]
This paper suggests a continual deepfake detection benchmark (CDDB) over a new collection of deepfakes from both known and unknown generative models.
We exploit multiple approaches to adapt multiclass incremental learning methods, commonly used in the continual visual recognition, to the continual deepfake detection problem.
arXiv Detail & Related papers (2022-05-11T13:07:19Z) - On Improving Cross-dataset Generalization of Deepfake Detectors [1.0152838128195467]
Facial manipulation by deep fake has caused major security risks and raised severe societal concerns.
We formulate deep fake detection as a hybrid combination of supervised and reinforcement learning (RL) to improve its cross-dataset generalization performance.
We demonstrate the superiority of our method over existing published research in cross-dataset generalization of deep fake detectors, thus obtaining state-of-the-art performance.
arXiv Detail & Related papers (2022-04-08T20:34:53Z) - Impact of Benign Modifications on Discriminative Performance of Deepfake
Detectors [11.881119750753648]
A large number of deepfake detectors have been proposed recently in order to identify such content.
Deepfakes are increasingly popular in both good faith applications such as in entertainment and maliciously intended manipulations such as in image and video forgery.
This paper proposes a more rigorous and systematic framework to assess the performance of deepfake detectors in more realistic situations.
arXiv Detail & Related papers (2021-11-14T22:50:39Z) - M2TR: Multi-modal Multi-scale Transformers for Deepfake Detection [74.19291916812921]
forged images generated by Deepfake techniques pose a serious threat to the trustworthiness of digital information.
In this paper, we aim to capture the subtle manipulation artifacts at different scales for Deepfake detection.
We introduce a high-quality Deepfake dataset, SR-DF, which consists of 4,000 DeepFake videos generated by state-of-the-art face swapping and facial reenactment methods.
arXiv Detail & Related papers (2021-04-20T05:43:44Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.