Robust AI-Generated Face Detection with Imbalanced Data

• URL: http://arxiv.org/abs/2505.02182v1
• Date: Sun, 04 May 2025 17:02:10 GMT
• Title: Robust AI-Generated Face Detection with Imbalanced Data
• Authors: Yamini Sri Krubha, Aryana Hou, Braden Vester, Web Walker, Xin Wang, Li Lin, Shu Hu,
• Abstract summary: Current deepfake detection techniques have evolved from CNN-based methods focused on local artifacts to more advanced approaches using vision transformers and multimodal models like CLIP.<n>Despite recent progress, state-of-the-art deepfake detectors still face major challenges in handling distribution shifts from emerging generative models.<n>We propose a framework that combines dynamic loss reweighting and ranking-based optimization, which achieves superior generalization and performance under imbalanced dataset conditions.
• Score: 10.360215701635674
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deepfakes, created using advanced AI techniques such as Variational Autoencoder and Generative Adversarial Networks, have evolved from research and entertainment applications into tools for malicious activities, posing significant threats to digital trust. Current deepfake detection techniques have evolved from CNN-based methods focused on local artifacts to more advanced approaches using vision transformers and multimodal models like CLIP, which capture global anomalies and improve cross-domain generalization. Despite recent progress, state-of-the-art deepfake detectors still face major challenges in handling distribution shifts from emerging generative models and addressing severe class imbalance between authentic and fake samples in deepfake datasets, which limits their robustness and detection accuracy. To address these challenges, we propose a framework that combines dynamic loss reweighting and ranking-based optimization, which achieves superior generalization and performance under imbalanced dataset conditions. The code is available at https://github.com/Purdue-M2/SP_CUP.

Related papers

• Enhancing Abnormality Identification: Robust Out-of-Distribution Strategies for Deepfake Detection [2.4851820343103035]
  We propose two novel Out-Of-Distribution (OOD) detection approaches.<n>The first approach is trained to reconstruct the input image, while the second incorporates an attention mechanism for detecting OODs.<n>Our method achieves promising results in deepfake detection and ranks among the top-performing configurations on the benchmark.
  arXiv  Detail & Related papers  (2025-06-03T13:24:33Z)
• Cross-Branch Orthogonality for Improved Generalization in Face Deepfake Detection [43.2796409299818]
  Deepfakes are becoming a nuisance to law enforcement authorities and the general public.<n>Existing deepfake detectors are struggling to keep up with the pace of improvements in deepfake generation.<n>This paper proposes a new strategy that leverages coarse-to-fine spatial information, semantic information, and their interactions.
  arXiv  Detail & Related papers  (2025-05-08T01:49:53Z)
• Comparative Analysis of Deepfake Detection Models: New Approaches and Perspectives [0.0]
  This work investigates and compares different approaches for identifying deepfakes, focusing on the GenConViT model.<n>To contextualize the research, the social and legal impacts of deepfakes are addressed, as well as the technical fundamentals of their creation and detection.<n>The obtained results indicated that GenConViT, after fine-tuning, exhibited superior performance in terms of accuracy (93.82%) and generalization capacity.
  arXiv  Detail & Related papers  (2025-04-03T02:10:27Z)
• Adaptive Meta-Learning for Robust Deepfake Detection: A Multi-Agent Framework to Data Drift and Model Generalization [6.589206192038365]
  This paper proposes an adversarial meta-learning algorithm using task-specific adaptive sample synthesis and consistency regularization. It boosts both robustness and generalization of the model. Experimental results demonstrate the model's consistent performance across various datasets, outperforming the models in comparison.
  arXiv  Detail & Related papers  (2024-11-12T19:55:07Z)
• Comprehensive Botnet Detection by Mitigating Adversarial Attacks, Navigating the Subtleties of Perturbation Distances and Fortifying Predictions with Conformal Layers [1.6001193161043425]
  Botnets are computer networks controlled by malicious actors that present significant cybersecurity challenges. This research addresses the sophisticated adversarial manipulations posed by attackers, aiming to undermine machine learning-based botnet detection systems. We introduce a flow-based detection approach, leveraging machine learning and deep learning algorithms trained on the ISCX and ISOT datasets.
  arXiv  Detail & Related papers  (2024-09-01T08:53:21Z)
• Open-Set Deepfake Detection: A Parameter-Efficient Adaptation Method with Forgery Style Mixture [58.60915132222421]
  We introduce an approach that is both general and parameter-efficient for face forgery detection. We design a forgery-style mixture formulation that augments the diversity of forgery source domains. We show that the designed model achieves state-of-the-art generalizability with significantly reduced trainable parameters.
  arXiv  Detail & Related papers  (2024-08-23T01:53:36Z)
• Towards More General Video-based Deepfake Detection through Facial Component Guided Adaptation for Foundation Model [16.69101880602321]
  We propose a novel side-network-based decoder for generalized video-based Deepfake detection.<n>We also introduce Facial Component Guidance (FCG) to enhance spatial learning generalizability.<n>Our approach demonstrates promising generalizability on challenging Deepfake datasets.
  arXiv  Detail & Related papers  (2024-04-08T14:58:52Z)
• Deepfake Sentry: Harnessing Ensemble Intelligence for Resilient Detection and Generalisation [0.8796261172196743]
  We propose a proactive and sustainable deepfake training augmentation solution. We employ a pool of autoencoders that mimic the effect of the artefacts introduced by the deepfake generator models. Experiments reveal that our proposed ensemble autoencoder-based data augmentation learning approach offers improvements in terms of generalisation.
  arXiv  Detail & Related papers  (2024-03-29T19:09:08Z)
• GenFace: A Large-Scale Fine-Grained Face Forgery Benchmark and Cross Appearance-Edge Learning [50.7702397913573]
  The rapid advancement of photorealistic generators has reached a critical juncture where the discrepancy between authentic and manipulated images is increasingly indistinguishable. Although there have been a number of publicly available face forgery datasets, the forgery faces are mostly generated using GAN-based synthesis technology. We propose a large-scale, diverse, and fine-grained high-fidelity dataset, namely GenFace, to facilitate the advancement of deepfake detection.
  arXiv  Detail & Related papers  (2024-02-03T03:13:50Z)
• Towards General Visual-Linguistic Face Forgery Detection [95.73987327101143]
  Deepfakes are realistic face manipulations that can pose serious threats to security, privacy, and trust. Existing methods mostly treat this task as binary classification, which uses digital labels or mask signals to train the detection model. We propose a novel paradigm named Visual-Linguistic Face Forgery Detection(VLFFD), which uses fine-grained sentence-level prompts as the annotation.
  arXiv  Detail & Related papers  (2023-07-31T10:22:33Z)
• Semantic Perturbations with Normalizing Flows for Improved Generalization [62.998818375912506]
  We show that perturbations in the latent space can be used to define fully unsupervised data augmentations. We find that our latent adversarial perturbations adaptive to the classifier throughout its training are most effective.
  arXiv  Detail & Related papers  (2021-08-18T03:20:00Z)
• Anomaly Detection Based on Selection and Weighting in Latent Space [73.01328671569759]
  We propose a novel selection-and-weighting-based anomaly detection framework called SWAD. Experiments on both benchmark and real-world datasets have shown the effectiveness and superiority of SWAD.
  arXiv  Detail & Related papers  (2021-03-08T10:56:38Z)
• Artificial Fingerprinting for Generative Models: Rooting Deepfake Attribution in Training Data [64.65952078807086]
  Photorealistic image generation has reached a new level of quality due to the breakthroughs of generative adversarial networks (GANs) Yet, the dark side of such deepfakes, the malicious use of generated media, raises concerns about visual misinformation. We seek a proactive and sustainable solution on deepfake detection by introducing artificial fingerprints into the models.
  arXiv  Detail & Related papers  (2020-07-16T16:49:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.