AUDDT: Audio Unified Deepfake Detection Benchmark Toolkit

• URL: http://arxiv.org/abs/2509.21597v1
• Date: Thu, 25 Sep 2025 21:09:40 GMT
• Title: AUDDT: Audio Unified Deepfake Detection Benchmark Toolkit
• Authors: Yi Zhu, Heitor R. Guimarães, Arthur Pimentel, Tiago Falk,
• Abstract summary: We systematically review 28 existing audio deepfake datasets and present an open-source benchmarking toolkit called AUDDT.<n>The goal of this toolkit is to automate the evaluation of pretrained detectors across these 28 datasets, giving users direct feedback on the advantages and shortcomings of their deepfake detectors.
• Score: 7.279026980203529
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With the prevalence of artificial intelligence (AI)-generated content, such as audio deepfakes, a large body of recent work has focused on developing deepfake detection techniques. However, most models are evaluated on a narrow set of datasets, leaving their generalization to real-world conditions uncertain. In this paper, we systematically review 28 existing audio deepfake datasets and present an open-source benchmarking toolkit called AUDDT (https://github.com/MuSAELab/AUDDT). The goal of this toolkit is to automate the evaluation of pretrained detectors across these 28 datasets, giving users direct feedback on the advantages and shortcomings of their deepfake detectors. We start by showcasing the usage of the developed toolkit, the composition of our benchmark, and the breakdown of different deepfake subgroups. Next, using a widely adopted pretrained deepfake detector, we present in- and out-of-domain detection results, revealing notable differences across conditions and audio manipulation types. Lastly, we also analyze the limitations of these existing datasets and their gap relative to practical deployment scenarios.

Related papers

• Speech DF Arena: A Leaderboard for Speech DeepFake Detection Models [11.803603620762486]
  Speech DeepFake (DF) Arena is the first comprehensive benchmark for audio deepfake detection.<n>DF Arena provides a toolkit to uniformly evaluate detection systems, currently across 14 diverse datasets and attack scenarios.<n>It also includes a leaderboard to compare and rank the systems to help researchers and developers enhance their reliability and robustness.
  arXiv  Detail & Related papers  (2025-09-02T22:11:29Z)
• Contextual Cross-Modal Attention for Audio-Visual Deepfake Detection and Localization [3.9440964696313485]
  In the digital age, the emergence of deepfakes and synthetic media presents a significant threat to societal and political integrity. Deepfakes based on multi-modal manipulation, such as audio-visual, are more realistic and pose a greater threat. We propose a novel multi-modal attention framework based on recurrent neural networks (RNNs) that leverages contextual information for audio-visual deepfake detection.
  arXiv  Detail & Related papers  (2024-08-02T18:45:01Z)
• Bayesian Detector Combination for Object Detection with Crowdsourced Annotations [49.43709660948812]
  Acquiring fine-grained object detection annotations in unconstrained images is time-consuming, expensive, and prone to noise. We propose a novel Bayesian Detector Combination (BDC) framework to more effectively train object detectors with noisy crowdsourced annotations. BDC is model-agnostic, requires no prior knowledge of the annotators' skill level, and seamlessly integrates with existing object detection models.
  arXiv  Detail & Related papers  (2024-07-10T18:00:54Z)
• DF40: Toward Next-Generation Deepfake Detection [62.073997142001424]
  existing works identify top-notch detection algorithms and models by adhering to the common practice: training detectors on one specific dataset and testing them on other prevalent deepfake datasets. But can these stand-out "winners" be truly applied to tackle the myriad of realistic and diverse deepfakes lurking in the real world? We construct a highly diverse deepfake detection dataset called DF40, which comprises 40 distinct deepfake techniques.
  arXiv  Detail & Related papers  (2024-06-19T12:35:02Z)
• Integrating Audio-Visual Features for Multimodal Deepfake Detection [33.51027054306748]
  Deepfakes are AI-generated media in which an image or video has been digitally modified. This paper proposes an audio-visual-based method for deepfake detection, which integrates fine-grained deepfake identification with binary classification.
  arXiv  Detail & Related papers  (2023-10-05T18:19:56Z)
• CrossDF: Improving Cross-Domain Deepfake Detection with Deep Information Decomposition [53.860796916196634]
  We propose a Deep Information Decomposition (DID) framework to enhance the performance of Cross-dataset Deepfake Detection (CrossDF) Unlike most existing deepfake detection methods, our framework prioritizes high-level semantic features over specific visual artifacts. It adaptively decomposes facial features into deepfake-related and irrelevant information, only using the intrinsic deepfake-related information for real/fake discrimination.
  arXiv  Detail & Related papers  (2023-09-30T12:30:25Z)
• DeepfakeBench: A Comprehensive Benchmark of Deepfake Detection [55.70982767084996]
  A critical yet frequently overlooked challenge in the field of deepfake detection is the lack of a standardized, unified, comprehensive benchmark. We present the first comprehensive benchmark for deepfake detection, called DeepfakeBench, which offers three key contributions. DeepfakeBench contains 15 state-of-the-art detection methods, 9CL datasets, a series of deepfake detection evaluation protocols and analysis tools, as well as comprehensive evaluations.
  arXiv  Detail & Related papers  (2023-07-04T01:34:41Z)
• A Continual Deepfake Detection Benchmark: Dataset, Methods, and Essentials [97.69553832500547]
  This paper suggests a continual deepfake detection benchmark (CDDB) over a new collection of deepfakes from both known and unknown generative models. We exploit multiple approaches to adapt multiclass incremental learning methods, commonly used in the continual visual recognition, to the continual deepfake detection problem.
  arXiv  Detail & Related papers  (2022-05-11T13:07:19Z)
• Voice-Face Homogeneity Tells Deepfake [56.334968246631725]
  Existing detection approaches contribute to exploring the specific artifacts in deepfake videos. We propose to perform the deepfake detection from an unexplored voice-face matching view. Our model obtains significantly improved performance as compared to other state-of-the-art competitors.
  arXiv  Detail & Related papers  (2022-03-04T09:08:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.