Towards Generalizable Deepfake Image Detection with Vision Transformers

• URL: http://arxiv.org/abs/2604.17376v1
• Date: Sun, 19 Apr 2026 10:59:17 GMT
• Title: Towards Generalizable Deepfake Image Detection with Vision Transformers
• Abstract summary: In this paper, we use an ensemble of fine-tuned vision transformers like DINOv2, AIMv2 and OpenCLIP's ViT-L/14 to create a generalizable method to detect deepfakes.<n> Experimental results show that our ensemble outperforms individual models and strong CNN baselines, achieving an AUC of 96.77% and an Equal Error Rate (EER) of just 9% on the DF-Wild test set.
• Score: 0.7641495778058912
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: In today's day and age, we face a challenge in detecting deepfake images because of the fast evolution of modern generative models and the poor generalization capability of existing methods. In this paper, we use an ensemble of fine-tuned vision transformers like DINOv2, AIMv2 and OpenCLIP's ViT-L/14 to create generalizable method to detect deepfakes. We use the DF-Wild dataset released as part of the IEEE SP Cup 2025, because it uses a challenging and diverse set of manipulations and generation techniques. We started our experiments with CNN classifiers trained on spatial features. Experimental results show that our ensemble outperforms individual models and strong CNN baselines, achieving an AUC of 96.77% and an Equal Error Rate (EER) of just 9% on the DF-Wild test set, beating the state-of-the-art deepfake detection algorithm Effort by 7.05% and 8% in AUC and EER respectively. This was the winning solution for SP Cup, presented at ICASSP 2025.

Related papers

• The Alpha Blending Hypothesis: Compositing Shortcut in Deepfake Detection [48.85953407706351]
  We introduce the Alpha Blending Hypothesis, positing that state-of-the-art frame-based detectors primarily function as alpha blending searchers.<n>We experimentally validate the hypothesis, demonstrating that deepfake detectors exhibit high sensitivity to the so-called self-blended images.<n>We show that predictions from explicit blending searchers and models resilient to blending shortcuts are highly complementary, yielding a state-of-the-art AUROC of 94.0% in an ensemble configuration.
  arXiv  Detail & Related papers  (2026-05-11T10:35:39Z)
• Practical Manipulation Model for Robust Deepfake Detection [55.2480439325792]
  We develop a more real-world degradation model in the area of image super-resolution.<n>We extend the space of pseudo-fakes by using Poisson blending, more diverse masks, generator artifacts, and distractors.<n>We show clear increases of $3.51%$ and $6.21%$ AUC on the DFDC and DFDCP datasets, respectively.
  arXiv  Detail & Related papers  (2025-06-05T15:06:16Z)
• CAE-Net: Generalized Deepfake Image Detection using Convolution and Attention Mechanisms with Spatial and Frequency Domain Features [0.6700983301090583]
  We propose a disjoint set-based multistage training method to address the class imbalance and devised an ensemble-based architecture emphCAE-Net.<n>Our architecture consists of a convolution- and attention-based ensemble network, and employs three different neural network architectures.<n>Individually, the EfficientNet B0 architecture has achieved 90.79% accuracy, whereas the ConvNeXt and the DeiT architecture have achieved 89.49% and 89.32% accuracy, respectively.
  arXiv  Detail & Related papers  (2025-02-15T06:02:11Z)
• DFCon: Attention-Driven Supervised Contrastive Learning for Robust Deepfake Detection [0.3818645814949463]
  This report presents our approach for the IEEE SP Cup 2025: Deepfake Face Detection in the Wild (DFWild-Cup)<n>Our methodology employs advanced backbone models, including MaxViT, CoAtNet, and EVA-02, fine-tuned using supervised contrastive loss to enhance feature separation.<n>The proposed system addresses the challenges of detecting deepfakes in real-world conditions and achieves a commendable accuracy of 95.83% on the validation dataset.
  arXiv  Detail & Related papers  (2025-01-28T04:46:50Z)
• Data-Independent Operator: A Training-Free Artifact Representation Extractor for Generalizable Deepfake Detection [105.9932053078449]
  In this work, we show that, on the contrary, the small and training-free filter is sufficient to capture more general artifact representations. Due to its unbias towards both the training and test sources, we define it as Data-Independent Operator (DIO) to achieve appealing improvements on unseen sources. Our detector achieves a remarkable improvement of $13.3%$, establishing a new state-of-the-art performance.
  arXiv  Detail & Related papers  (2024-03-11T15:22:28Z)
• ViTAEv2: Vision Transformer Advanced by Exploring Inductive Bias for Image Recognition and Beyond [76.35955924137986]
  We propose a Vision Transformer Advanced by Exploring intrinsic IB from convolutions, i.e., ViTAE. ViTAE has several spatial pyramid reduction modules to downsample and embed the input image into tokens with rich multi-scale context. We obtain the state-of-the-art classification performance, i.e., 88.5% Top-1 classification accuracy on ImageNet validation set and the best 91.2% Top-1 accuracy on ImageNet real validation set.
  arXiv  Detail & Related papers  (2022-02-21T10:40:05Z)
• Combining EfficientNet and Vision Transformers for Video Deepfake Detection [6.365889364810238]
  Deepfakes are the result of digital manipulation to obtain credible videos in order to deceive the viewer. In this study, we combine various types of Vision Transformers with a convolutional EfficientNet B0 used as a feature extractor. The best model achieved an AUC of 0.951 and an F1 score of 88.0%, very close to the state-of-the-art on the DeepFake Detection Challenge (DFDC)
  arXiv  Detail & Related papers  (2021-07-06T13:35:11Z)
• M2TR: Multi-modal Multi-scale Transformers for Deepfake Detection [74.19291916812921]
  forged images generated by Deepfake techniques pose a serious threat to the trustworthiness of digital information. In this paper, we aim to capture the subtle manipulation artifacts at different scales for Deepfake detection. We introduce a high-quality Deepfake dataset, SR-DF, which consists of 4,000 DeepFake videos generated by state-of-the-art face swapping and facial reenactment methods.
  arXiv  Detail & Related papers  (2021-04-20T05:43:44Z)
• Deepfake Detection Scheme Based on Vision Transformer and Distillation [4.716110829725784]
  We propose a Vision Transformer model with distillation methodology for detecting fake videos. We verify that the proposed scheme with patch embedding as input outperforms the state-of-the-art using the combined CNN features.
  arXiv  Detail & Related papers  (2021-04-03T09:13:05Z)
• Deepfake Video Detection Using Convolutional Vision Transformer [0.0]
  Deep learning techniques can generate and synthesis hyper-realistic videos known as Deepfakes. Deepfakes pose a looming threat to everyone if used for harmful purposes such as identity theft, phishing, and scam. We propose a Convolutional Vision Transformer for the detection of Deepfakes.
  arXiv  Detail & Related papers  (2021-02-22T15:56:05Z)
• Adversarially robust deepfake media detection using fused convolutional neural network predictions [79.00202519223662]
  Current deepfake detection systems struggle against unseen data. We employ three different deep Convolutional Neural Network (CNN) models to classify fake and real images extracted from videos. The proposed technique outperforms state-of-the-art models with 96.5% accuracy.
  arXiv  Detail & Related papers  (2021-02-11T11:28:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.