Classifying Deepfakes Using Swin Transformers

• URL: http://arxiv.org/abs/2501.15656v2
• Date: Fri, 31 Jan 2025 16:16:30 GMT
• Title: Classifying Deepfakes Using Swin Transformers
• Authors: Aprille J. Xi, Eason Chen,
• Abstract summary: This study explores the application of Swin Transformers, a state-of-the-art architecture leveraging shifted windows for self-attention, in detecting and classifying deepfake images.<n>We evaluate the Swin Transformer and hybrid models such as Swin-ResNet and Swin-KNN, focusing on their ability to identify subtle manipulation artifacts.
• Score: 12.693895808318794
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The proliferation of deepfake technology poses significant challenges to the authenticity and trustworthiness of digital media, necessitating the development of robust detection methods. This study explores the application of Swin Transformers, a state-of-the-art architecture leveraging shifted windows for self-attention, in detecting and classifying deepfake images. Using the Real and Fake Face Detection dataset by Yonsei University's Computational Intelligence Photography Lab, we evaluate the Swin Transformer and hybrid models such as Swin-ResNet and Swin-KNN, focusing on their ability to identify subtle manipulation artifacts. Our results demonstrate that the Swin Transformer outperforms conventional CNN-based architectures, including VGG16, ResNet18, and AlexNet, achieving a test accuracy of 71.29%. Additionally, we present insights into hybrid model design, highlighting the complementary strengths of transformer and CNN-based approaches in deepfake detection. This study underscores the potential of transformer-based architectures for improving accuracy and generalizability in image-based manipulation detection, paving the way for more effective countermeasures against deepfake threats.

Related papers

• Comparative Analysis of Deepfake Detection Models: New Approaches and Perspectives [0.0]
  This work investigates and compares different approaches for identifying deepfakes, focusing on the GenConViT model. To contextualize the research, the social and legal impacts of deepfakes are addressed, as well as the technical fundamentals of their creation and detection. The obtained results indicated that GenConViT, after fine-tuning, exhibited superior performance in terms of accuracy (93.82%) and generalization capacity.
  arXiv  Detail & Related papers  (2025-04-03T02:10:27Z)
• Inverting Visual Representations with Detection Transformers [0.8124699127636158]
  We apply the approach of training inverse models to reconstruct input images from intermediate layers within a Detection Transformer.<n>We demonstrate critical properties of Detection Transformers, including contextual shape robustness, inter-layer correlation, and preservation to color perturbations.
  arXiv  Detail & Related papers  (2024-12-09T14:43:06Z)
• Addressing Vulnerabilities in AI-Image Detection: Challenges and Proposed Solutions [0.0]
  This study evaluates the effectiveness of convolutional neural networks (CNNs) and DenseNet architectures for detecting AI-generated images.<n>We analyze the impact of updates and modifications such as Gaussian blurring, prompt text changes, and Low-Rank Adaptation (LoRA) on detection accuracy.<n>The findings highlight vulnerabilities in current detection methods and propose strategies to enhance the robustness and reliability of AI-image detection systems.
  arXiv  Detail & Related papers  (2024-11-26T06:35:26Z)
• Deepfake Sentry: Harnessing Ensemble Intelligence for Resilient Detection and Generalisation [0.8796261172196743]
  We propose a proactive and sustainable deepfake training augmentation solution. We employ a pool of autoencoders that mimic the effect of the artefacts introduced by the deepfake generator models. Experiments reveal that our proposed ensemble autoencoder-based data augmentation learning approach offers improvements in terms of generalisation.
  arXiv  Detail & Related papers  (2024-03-29T19:09:08Z)
• GenFace: A Large-Scale Fine-Grained Face Forgery Benchmark and Cross Appearance-Edge Learning [50.7702397913573]
  The rapid advancement of photorealistic generators has reached a critical juncture where the discrepancy between authentic and manipulated images is increasingly indistinguishable. Although there have been a number of publicly available face forgery datasets, the forgery faces are mostly generated using GAN-based synthesis technology. We propose a large-scale, diverse, and fine-grained high-fidelity dataset, namely GenFace, to facilitate the advancement of deepfake detection.
  arXiv  Detail & Related papers  (2024-02-03T03:13:50Z)
• Generalized Deepfakes Detection with Reconstructed-Blended Images and Multi-scale Feature Reconstruction Network [14.749857283918157]
  We present a blended-based detection approach that has robust applicability to unseen datasets. Experiments demonstrated that this approach results in better performance in both cross-manipulation detection and cross-dataset detection on unseen data.
  arXiv  Detail & Related papers  (2023-12-13T09:49:15Z)
• DeepFidelity: Perceptual Forgery Fidelity Assessment for Deepfake Detection [67.3143177137102]
  Deepfake detection refers to detecting artificially generated or edited faces in images or videos. We propose a novel Deepfake detection framework named DeepFidelity to adaptively distinguish real and fake faces.
  arXiv  Detail & Related papers  (2023-12-07T07:19:45Z)
• Deep Convolutional Pooling Transformer for Deepfake Detection [54.10864860009834]
  We propose a deep convolutional Transformer to incorporate decisive image features both locally and globally. Specifically, we apply convolutional pooling and re-attention to enrich the extracted features and enhance efficacy. The proposed solution consistently outperforms several state-of-the-art baselines on both within- and cross-dataset experiments.
  arXiv  Detail & Related papers  (2022-09-12T15:05:41Z)
• M2TR: Multi-modal Multi-scale Transformers for Deepfake Detection [74.19291916812921]
  forged images generated by Deepfake techniques pose a serious threat to the trustworthiness of digital information. In this paper, we aim to capture the subtle manipulation artifacts at different scales for Deepfake detection. We introduce a high-quality Deepfake dataset, SR-DF, which consists of 4,000 DeepFake videos generated by state-of-the-art face swapping and facial reenactment methods.
  arXiv  Detail & Related papers  (2021-04-20T05:43:44Z)
• Artificial Fingerprinting for Generative Models: Rooting Deepfake Attribution in Training Data [64.65952078807086]
  Photorealistic image generation has reached a new level of quality due to the breakthroughs of generative adversarial networks (GANs) Yet, the dark side of such deepfakes, the malicious use of generated media, raises concerns about visual misinformation. We seek a proactive and sustainable solution on deepfake detection by introducing artificial fingerprints into the models.
  arXiv  Detail & Related papers  (2020-07-16T16:49:55Z)
• DeepFake Detection by Analyzing Convolutional Traces [0.0]
  We focus on the analysis of Deepfakes of human faces with the objective of creating a new detection method. The proposed technique, by means of an Expectation Maximization (EM) algorithm, extracts a set of local features specifically addressed to model the underlying convolutional generative process. Results demonstrated the effectiveness of the technique in distinguishing the different architectures and the corresponding generation process.
  arXiv  Detail & Related papers  (2020-04-22T09:02:55Z)
• Robustness Verification for Transformers [165.25112192811764]
  We develop the first robustness verification algorithm for Transformers. The certified robustness bounds computed by our method are significantly tighter than those by naive Interval Bound propagation. These bounds also shed light on interpreting Transformers as they consistently reflect the importance of different words in sentiment analysis.
  arXiv  Detail & Related papers  (2020-02-16T17:16:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.