Improving Cross-dataset Deepfake Detection with Deep Information Decomposition

• URL: http://arxiv.org/abs/2310.00359v1
• Date: Sat, 30 Sep 2023 12:30:25 GMT
• Title: Improving Cross-dataset Deepfake Detection with Deep Information Decomposition
• Authors: Shanmin Yang, Shu Hu, Bin Zhu, Ying Fu, Siwei Lyu, Xi Wu, Xin Wang
• Abstract summary: Deepfake technology poses a significant threat to security and social trust. Existing detection methods suffer from sharp performance degradation when faced with cross-dataset scenarios. We propose a deep information decomposition (DID) framework in this paper.
• Score: 57.284370468207214
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deepfake technology poses a significant threat to security and social trust. Although existing detection methods have demonstrated high performance in identifying forgeries within datasets using the same techniques for training and testing, they suffer from sharp performance degradation when faced with cross-dataset scenarios where unseen deepfake techniques are tested. To address this challenge, we propose a deep information decomposition (DID) framework in this paper. Unlike most existing deepfake detection methods, our framework prioritizes high-level semantic features over visual artifacts. Specifically, it decomposes facial features into deepfake-related and irrelevant information and optimizes the deepfake information for real/fake discrimination to be independent of other factors. Our approach improves the robustness of deepfake detection against various irrelevant information changes and enhances the generalization ability of the framework to detect unseen forgery methods. Extensive experimental comparisons with existing state-of-the-art detection methods validate the effectiveness and superiority of the DID framework on cross-dataset deepfake detection.

Related papers

• DiffusionFF: Face Forgery Detection via Diffusion-based Artifact Localization [21.139016641596676]
  DiffusionFF is a novel framework that enhances face forgery detection through diffusion-based artifact localization.<n>Our method utilizes a denoising diffusion model to generate high-quality Structural Dissimilarity (DSSIM) maps, which effectively capture subtle traces of manipulation.
  arXiv  Detail & Related papers  (2025-08-03T18:06:04Z)
• Comparative Analysis of Deepfake Detection Models: New Approaches and Perspectives [0.0]
  This work investigates and compares different approaches for identifying deepfakes, focusing on the GenConViT model. To contextualize the research, the social and legal impacts of deepfakes are addressed, as well as the technical fundamentals of their creation and detection. The obtained results indicated that GenConViT, after fine-tuning, exhibited superior performance in terms of accuracy (93.82%) and generalization capacity.
  arXiv  Detail & Related papers  (2025-04-03T02:10:27Z)
• Securing Social Media Against Deepfakes using Identity, Behavioral, and Geometric Signatures [6.3947036687002985]
  Trust in social media is a growing concern due to its ability to influence significant societal changes. Deepfake multimedia undermine the authenticity of shared content. Existing detection techniques tend to perform well only on specific types of deepfakes they were trained on.
  arXiv  Detail & Related papers  (2024-12-07T01:17:21Z)
• Leveraging Mixture of Experts for Improved Speech Deepfake Detection [53.69740463004446]
  Speech deepfakes pose a significant threat to personal security and content authenticity. We introduce a novel approach for enhancing speech deepfake detection performance using a Mixture of Experts architecture.
  arXiv  Detail & Related papers  (2024-09-24T13:24:03Z)
• UniForensics: Face Forgery Detection via General Facial Representation [60.5421627990707]
  High-level semantic features are less susceptible to perturbations and not limited to forgery-specific artifacts, thus having stronger generalization. We introduce UniForensics, a novel deepfake detection framework that leverages a transformer-based video network, with a meta-functional face classification for enriched facial representation.
  arXiv  Detail & Related papers  (2024-07-26T20:51:54Z)
• Explicit Correlation Learning for Generalizable Cross-Modal Deepfake Detection [33.20064862916194]
  This paper aims to learn potential cross-modal correlation to enhance deepfake detection towards various generation scenarios. Our approach introduces a correlation distillation task, which models the inherent cross-modal correlation based on content information. We present the Cross-Modal Deepfake dataset with four generation methods to evaluate the detection of diverse cross-modal deepfakes.
  arXiv  Detail & Related papers  (2024-04-30T00:25:44Z)
• Adversarially Robust Deepfake Detection via Adversarial Feature Similarity Learning [0.0]
  Deepfake technology has raised concerns about the authenticity of digital content, necessitating the development of effective detection methods. Adversaries can manipulate deepfake videos with small, imperceptible perturbations that can deceive the detection models into producing incorrect outputs. We introduce Adversarial Feature Similarity Learning (AFSL), which integrates three fundamental deep feature learning paradigms.
  arXiv  Detail & Related papers  (2024-02-06T11:35:05Z)
• DeepFidelity: Perceptual Forgery Fidelity Assessment for Deepfake Detection [67.3143177137102]
  Deepfake detection refers to detecting artificially generated or edited faces in images or videos. We propose a novel Deepfake detection framework named DeepFidelity to adaptively distinguish real and fake faces.
  arXiv  Detail & Related papers  (2023-12-07T07:19:45Z)
• Facial Forgery-based Deepfake Detection using Fine-Grained Features [7.378937711027777]
  Facial forgery by deepfakes has caused major security risks and raised severe societal concerns. We formulate deepfake detection as a fine-grained classification problem and propose a new fine-grained solution to it. Our method is based on learning subtle and generalizable features by effectively suppressing background noise and learning discriminative features at various scales for deepfake detection.
  arXiv  Detail & Related papers  (2023-10-10T21:30:05Z)
• Deep Convolutional Pooling Transformer for Deepfake Detection [54.10864860009834]
  We propose a deep convolutional Transformer to incorporate decisive image features both locally and globally. Specifically, we apply convolutional pooling and re-attention to enrich the extracted features and enhance efficacy. The proposed solution consistently outperforms several state-of-the-art baselines on both within- and cross-dataset experiments.
  arXiv  Detail & Related papers  (2022-09-12T15:05:41Z)
• On Improving Cross-dataset Generalization of Deepfake Detectors [1.0152838128195467]
  Facial manipulation by deep fake has caused major security risks and raised severe societal concerns. We formulate deep fake detection as a hybrid combination of supervised and reinforcement learning (RL) to improve its cross-dataset generalization performance. We demonstrate the superiority of our method over existing published research in cross-dataset generalization of deep fake detectors, thus obtaining state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-04-08T20:34:53Z)
• An Experimental Evaluation on Deepfake Detection using Deep Face Recognition [0.0]
  Deep learning has led to the generation of very realistic fake content, also known as deepfakes. Most of the current deepfake detection methods are deemed as a binary classification problem in distinguishing authentic images or videos from fake ones using two-class convolutional neural networks (CNNs) This paper thoroughly evaluate the efficacy of deep face recognition in identifying deepfakes, using different loss functions and deepfake generation techniques.
  arXiv  Detail & Related papers  (2021-10-04T18:02:56Z)
• M2TR: Multi-modal Multi-scale Transformers for Deepfake Detection [74.19291916812921]
  forged images generated by Deepfake techniques pose a serious threat to the trustworthiness of digital information. In this paper, we aim to capture the subtle manipulation artifacts at different scales for Deepfake detection. We introduce a high-quality Deepfake dataset, SR-DF, which consists of 4,000 DeepFake videos generated by state-of-the-art face swapping and facial reenactment methods.
  arXiv  Detail & Related papers  (2021-04-20T05:43:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.