Ethical and Technical Limits of Deepfake Speech Datasets

• URL: http://arxiv.org/abs/2606.10911v1
• Date: Tue, 09 Jun 2026 14:20:55 GMT
• Title: Ethical and Technical Limits of Deepfake Speech Datasets
• Abstract summary: We present a dataset-level audit of the deepfake speech landscape.<n>We examine key attributes including accessibility, documentation, demographic and language coverage, dataset scale, and the underlying bona fide speech sources.
• Score: 0.18665975431697432
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Claims about the robustness and fairness of deepfake speech detectors are only as credible as the datasets used to train and evaluate those systems. We present a dataset-level audit of the deepfake speech landscape. We compile and analyze 39 deepfake speech datasets, examining key attributes including accessibility, documentation, demographic and language coverage, dataset scale, and the underlying bona fide speech sources. Our audit reveals two important takeaways. Firstly, fairness assessment is largely infeasible because most datasets lack demographic metadata, and only a few contain gender or language labels. This prevents any meaningful subgroup analysis and leaves other demographic attributes unaddressed. Secondly, we identify substantial overlap in underlying bona fide source corpora across datasets, which can undermine cross-dataset evaluation and lead to overstated generalization claims.

Related papers

• TaigiSpeech: A Low-Resource Real-World Speech Intent Dataset and Preliminary Results with Scalable Data Mining In-the-Wild [102.11425887660327]
  Speech technologies have advanced rapidly and serve diverse populations worldwide.<n>Many languages remain underrepresented due to limited resources.<n>We introduce textbfTaigiSpeech, a real-world speech intent dataset in Taiwanese Taigi.
  arXiv  Detail & Related papers  (2026-03-23T01:44:45Z)
• Context and Transcripts Improve Detection of Deepfake Audios of Public Figures [24.44957433526574]
  Current audio deepfake detectors only analyze the audio file without considering either context or transcripts.<n>We show that sufficient context and/or the transcript can significantly improve the efficacy of audio deepfake detectors.<n>We additionally show that CADD, via its use of context and/or transcripts, is more robust to 5 adversarial evasion strategies.
  arXiv  Detail & Related papers  (2026-01-19T23:40:05Z)
• TalkVid: A Large-Scale Diversified Dataset for Audio-Driven Talking Head Synthesis [74.31705485094096]
  We introduce TalkVid, a new large-scale, high-quality, and diverse dataset containing 1244 hours of video from 7729 unique speakers.<n>TalkVid is curated through a principled, multi-stage automated pipeline that rigorously filters for motion stability, aesthetic quality, and facial detail.<n>We construct and release TalkVid-Bench, a stratified evaluation set of 500 clips meticulously balanced across key demographic and linguistic axes.
  arXiv  Detail & Related papers  (2025-08-19T08:31:15Z)
• SCDF: A Speaker Characteristics DeepFake Speech Dataset for Bias Analysis [1.2499537119440245]
  Speaker Characteristics Deepfake dataset contains over 237,000 utterances in a balanced representation of both male and female speakers.<n>We show that speaker characteristics significantly influence detection performance, revealing disparities across sex, language, age, and synthesizer type.<n>These findings highlight the need for bias-aware development and provide a foundation for building non-discriminatory deepfake detection systems.
  arXiv  Detail & Related papers  (2025-08-11T12:58:37Z)
• IndieFake Dataset: A Benchmark Dataset for Audio Deepfake Detection [0.4451479907610763]
  Deepfake technology offers benefits like AI assistants, better accessibility for speech impairments, and enhanced entertainment.<n>It also poses significant risks to security, privacy, and trust in digital communications.<n>Existing datasets lack diverse ethnic accents, making them inadequate for many real-world scenarios.<n>This work introduces the IndieFake dataset (IFD), featuring 27.17 hours of bonafide and deepfake audio from 50 English speaking Indian speakers.
  arXiv  Detail & Related papers  (2025-06-23T18:10:06Z)
• Bridging the Data Provenance Gap Across Text, Speech and Video [67.72097952282262]
  We conduct the largest and first-of-its-kind longitudinal audit across modalities of popular text, speech, and video datasets.<n>Our manual analysis covers nearly 4000 public datasets between 1990-2024, spanning 608 languages, 798 sources, 659 organizations, and 67 countries.<n>We find that multimodal machine learning applications have overwhelmingly turned to web-crawled, synthetic, and social media platforms, such as YouTube, for their training sets.
  arXiv  Detail & Related papers  (2024-12-19T01:30:19Z)
• Training-Free Deepfake Voice Recognition by Leveraging Large-Scale Pre-Trained Models [52.04189118767758]
  Generalization is a main issue for current audio deepfake detectors. In this paper we study the potential of large-scale pre-trained models for audio deepfake detection.
  arXiv  Detail & Related papers  (2024-05-03T15:27:11Z)
• SER_AMPEL: a multi-source dataset for speech emotion recognition of Italian older adults [58.49386651361823]
  SER_AMPEL is a multi-source dataset for speech emotion recognition (SER) It is collected with the aim of providing a reference for speech emotion recognition in case of Italian older adults. The evidence of the need for such a dataset emerges from the analysis of the state of the art.
  arXiv  Detail & Related papers  (2023-11-24T13:47:25Z)
• When a Language Question Is at Stake. A Revisited Approach to Label Sensitive Content [0.0]
  Article revisits an approach of pseudo-labeling sensitive data on the example of Ukrainian tweets covering the Russian-Ukrainian war. We provide a fundamental statistical analysis of the obtained data, evaluation of models used for pseudo-labelling, and set further guidelines on how the scientists can leverage the corpus.
  arXiv  Detail & Related papers  (2023-11-17T13:35:10Z)
• GBDF: Gender Balanced DeepFake Dataset Towards Fair DeepFake Detection [1.0152838128195467]
  Facial forgery by deepfakes has raised severe societal concerns. Recent studies have demonstrated that facial analysis-based deep learning models can discriminate based on protected attributes. It is vital to evaluate and understand the fairness of deepfake detectors across demographic variations such as gender and race.
  arXiv  Detail & Related papers  (2022-07-21T01:00:40Z)
• Algorithmic Fairness Datasets: the Story so Far [68.45921483094705]
  Data-driven algorithms are studied in diverse domains to support critical decisions, directly impacting people's well-being. A growing community of researchers has been investigating the equity of existing algorithms and proposing novel ones, advancing the understanding of risks and opportunities of automated decision-making for historically disadvantaged populations. Progress in fair Machine Learning hinges on data, which can be appropriately used only if adequately documented. Unfortunately, the algorithmic fairness community suffers from a collective data documentation debt caused by a lack of information on specific resources (opacity) and scatteredness of available information (sparsity)
  arXiv  Detail & Related papers  (2022-02-03T17:25:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.