When AUC Misleads: Polarization-Aware Evaluation of Deepfake Detectors under Domain Shift

• URL: http://arxiv.org/abs/2606.19184v1
• Date: Wed, 17 Jun 2026 15:24:06 GMT
• Title: When AUC Misleads: Polarization-Aware Evaluation of Deepfake Detectors under Domain Shift
• Abstract summary: Deepfake detection has become an active research area, with recent methods focusing on improving generalization to unseen manipulations.<n>We introduce a novel metric, Cross-dataset AUC (Cross-AUC), that averages per-domain AUCs with a measure of prediction polarization.<n>Cross-AUC not only assesses the generalization capabilities of deepfake detectors under domain shifts more realistically, but it is also interpretable as it better explains the reason behind a drop in performance.
• Score: 20.13000766160115
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Recent advances in generative AI, such as diffusion models and face-swapping tools, have enabled the creation of highly realistic deepfakes, leading to real-world harms including financial fraud and non-consensual explicit content. In response, deepfake detection has become an active research area, with recent methods increasingly focusing on improving generalization to unseen manipulations. This is typically evaluated using the Area Under the ROC Curve (AUC) measured separately across multiple datasets. However, such an evaluation fails to reflect real-world scenarios where detectors face a mixture of data sources and varying artifact types. To address this limitation, we introduce a novel metric, Cross-dataset AUC (Cross-AUC) that averages per-domain AUCs with a measure of prediction polarization for taking into account the robustness to domain shift. The polarization extent is quantified by the Wasserstein Distance between class score distributions. Cross-AUC not only assesses the generalization capabilities of deepfake detectors under domain shifts more realistically, but it is also interpretable as it better explains the reason behind a drop in performance. Experiments performed on seven benchmark datasets demonstrate its practical relevance.

Related papers

• Rethinking Cross-Domain Evaluation for Face Forgery Detection with Semantic Fine-grained Alignment and Mixture-of-Experts [22.1146135455996]
  Existing face forgery detectors can't achieve satisfactory performance because of poor generalization ability across datasets.<n>We propose textbfCross-AUC, an evaluation metric that can compute AUC across dataset pairs.<n>We also propose the novel framework textbfSemantic textbfFine-grained textbfAlignment and textbfMixture-of-Experts.
  arXiv  Detail & Related papers  (2026-04-23T09:39:33Z)
• DDL: A Large-Scale Datasets for Deepfake Detection and Localization in Diversified Real-World Scenarios [51.916287988122406]
  We present a novel large-scale deepfake detection and localization (textbfDDL) dataset containing over $textbf1.4M+$ forged samples.<n>Our DDL not only provides a more challenging benchmark for complex real-world forgeries but also offers crucial support for building next-generation deepfake detection, localization, and interpretability methods.
  arXiv  Detail & Related papers  (2025-06-29T15:29:03Z)
• Towards Open-world Generalized Deepfake Detection: General Feature Extraction via Unsupervised Domain Adaptation [15.737902253508235]
  Social platforms are flooded with vast amounts of unlabeled synthetic data and authentic data.<n>In open world scenarios, the amount of unlabeled data greatly exceeds that of labeled data.<n>We propose a novel Open-World Deepfake Detection Generalization Enhancement Training Strategy (OWG-DS) to improve the generalization ability of existing methods.
  arXiv  Detail & Related papers  (2025-05-18T10:12:12Z)
• Cross-Branch Orthogonality for Improved Generalization in Face Deepfake Detection [43.2796409299818]
  Deepfakes are becoming a nuisance to law enforcement authorities and the general public.<n>Existing deepfake detectors are struggling to keep up with the pace of improvements in deepfake generation.<n>This paper proposes a new strategy that leverages coarse-to-fine spatial information, semantic information, and their interactions.
  arXiv  Detail & Related papers  (2025-05-08T01:49:53Z)
• Cross-Domain Learning for Video Anomaly Detection with Limited Supervision [12.290352736331602]
  Video Anomaly Detection (VAD) automates the identification of unusual events, such as security threats in surveillance videos. Existing cross-domain VAD methods focus on unsupervised learning, resulting in performance that falls short of real-world expectations. We introduce a novel weakly-supervised framework for Cross-Domain Learning (CDL) in VAD that incorporates external data during training by estimating its prediction bias and adaptively minimizing that using the predicted uncertainty.
  arXiv  Detail & Related papers  (2024-08-09T17:28:29Z)
• DACAD: Domain Adaptation Contrastive Learning for Anomaly Detection in Multivariate Time Series [83.76994646443498]
  In time series anomaly detection, the scarcity of labeled data poses a challenge to the development of accurate models.<n>We propose a novel Domain Contrastive learning model for Anomaly Detection in time series (DACAD)<n>Our model employs supervised contrastive loss for the source domain and self-supervised contrastive triplet loss for the target domain.
  arXiv  Detail & Related papers  (2024-04-17T11:20:14Z)
• Learning Feature Inversion for Multi-class Anomaly Detection under General-purpose COCO-AD Benchmark [101.23684938489413]
  Anomaly detection (AD) is often focused on detecting anomalies for industrial quality inspection and medical lesion examination. This work first constructs a large-scale and general-purpose COCO-AD dataset by extending COCO to the AD field. Inspired by the metrics in the segmentation field, we propose several more practical threshold-dependent AD-specific metrics.
  arXiv  Detail & Related papers  (2024-04-16T17:38:26Z)
• CrossDF: Improving Cross-Domain Deepfake Detection with Deep Information Decomposition [53.860796916196634]
  We propose a Deep Information Decomposition (DID) framework to enhance the performance of Cross-dataset Deepfake Detection (CrossDF) Unlike most existing deepfake detection methods, our framework prioritizes high-level semantic features over specific visual artifacts. It adaptively decomposes facial features into deepfake-related and irrelevant information, only using the intrinsic deepfake-related information for real/fake discrimination.
  arXiv  Detail & Related papers  (2023-09-30T12:30:25Z)
• Towards Generalizable Deepfake Detection by Primary Region Regularization [52.41801719896089]
  This paper enhances the generalization capability from a novel regularization perspective. Our method consists of two stages, namely the static localization for primary region maps, and the dynamic exploitation of primary region masks. We conduct extensive experiments over three widely used deepfake datasets - DFDC, DF-1.0, and Celeb-DF with five backbones.
  arXiv  Detail & Related papers  (2023-07-24T05:43:34Z)
• On Improving Cross-dataset Generalization of Deepfake Detectors [1.0152838128195467]
  Facial manipulation by deep fake has caused major security risks and raised severe societal concerns. We formulate deep fake detection as a hybrid combination of supervised and reinforcement learning (RL) to improve its cross-dataset generalization performance. We demonstrate the superiority of our method over existing published research in cross-dataset generalization of deep fake detectors, thus obtaining state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-04-08T20:34:53Z)
• Cross-domain Object Detection through Coarse-to-Fine Feature Adaptation [62.29076080124199]
  This paper proposes a novel coarse-to-fine feature adaptation approach to cross-domain object detection. At the coarse-grained stage, foreground regions are extracted by adopting the attention mechanism, and aligned according to their marginal distributions. At the fine-grained stage, we conduct conditional distribution alignment of foregrounds by minimizing the distance of global prototypes with the same category but from different domains.
  arXiv  Detail & Related papers  (2020-03-23T13:40:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.