Gender Fairness in Audio Deepfake Detection: Performance and Disparity Analysis

• URL: http://arxiv.org/abs/2603.09007v1
• Date: Mon, 09 Mar 2026 22:52:12 GMT
• Title: Gender Fairness in Audio Deepfake Detection: Performance and Disparity Analysis
• Abstract summary: We have attempted a thorough analysis of gender dependent performance and fairness in audio deepfake detection models.<n>Our results show that even when the overall EER difference between genders appears low, fairness-aware evaluation reveals disparities in error distribution.<n>This work highlights the importance of fairness-aware evaluation for developing a more equitable, robust, and trustworthy audio deepfake detection system.
• Score: 2.000890150701116
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Audio deepfake detection aims to detect real human voices from those generated by Artificial Intelligence (AI) and has emerged as a significant problem in the field of voice biometrics systems. With the ever-improving quality of synthetic voice, the probability of such a voice being exploited for illicit practices like identity thest and impersonation increases. Although significant progress has been made in the field of Audio Deepfake Detection in recent times, the issue of gender bias remains underexplored and in its nascent stage In this paper, we have attempted a thorough analysis of gender dependent performance and fairness in audio deepfake detection models. We have used the ASVspoof 5 dataset and train a ResNet-18 classifier and evaluate detection performance across four different audio features, and compared the performance with baseline AASIST model. Beyond conventional metrics such as Equal Error Rate (EER %), we incorporated five established fairness metrics to quantify gender disparities in the model. Our results show that even when the overall EER difference between genders appears low, fairness-aware evaluation reveals disparities in error distribution that are obscured by aggregate performance measures. These findings demonstrate that reliance on standard metrics is unreliable, whereas fairness metrics provide critical insights into demographic-specific failure modes. This work highlights the importance of fairness-aware evaluation for developing a more equitable, robust, and trustworthy audio deepfake detection system.

Related papers

• A Mechanistic Analysis of Gender Sensitivity in Dense Retrieval Models [68.90383874606164]
  We analyze bi-encoder models to localize gender sensitivity, finding that the signal originates in input embeddings and propagates through a small set of late-layer attention heads.<n>Our findings provide a mechanistic basis for targeted debiasing and highlight the challenge of disentangling gender from relevance signals in shared model components.
  arXiv  Detail & Related papers  (2026-08-05T23:33:01Z)
• Towards Trustworthy Audio Deepfake Detection: A Systematic Framework for Diagnosing and Mitigating Gender Bias [2.000890150701116]
  We present the first diagnosis-first framework that identifies bias source before applying targeted mitigation.<n>Our diagnosis shows that bias does not stem from imbalanced training data but from acoustic representation differences.<n> Adjusting the decision threshold separately per gender reduces unfairness by 54% to 75% at no cost to detection accuracy.
  arXiv  Detail & Related papers  (2026-05-09T17:53:04Z)
• Walking Through Uncertainty: An Empirical Study of Uncertainty Estimation for Audio-Aware Large Language Models [54.041320081289996]
  We present the first systematic empirical study of uncertainty estimation in audio-aware large language models (ALLMs)<n>We benchmark five representative methods, including predictive entropy, length-normalized entropy, semantic entropy, discrete semantic entropy, and P(True)<n>Our results reveal two key findings. First, semantic-level and verification-based methods consistently outperform token-level baselines on general audio reasoning benchmarks.
  arXiv  Detail & Related papers  (2026-04-28T12:56:22Z)
• Bias in Gender Bias Benchmarks: How Spurious Features Distort Evaluation [116.86965910589775]
  We show that even minimal perturbations, such as masking just 10% of objects or weakly blurring backgrounds, can dramatically alter bias scores.<n>This suggests that current bias evaluations reflect model responses to spurious features rather than gender bias.
  arXiv  Detail & Related papers  (2025-09-09T11:14:11Z)
• SCDF: A Speaker Characteristics DeepFake Speech Dataset for Bias Analysis [1.2499537119440245]
  Speaker Characteristics Deepfake dataset contains over 237,000 utterances in a balanced representation of both male and female speakers.<n>We show that speaker characteristics significantly influence detection performance, revealing disparities across sex, language, age, and synthesizer type.<n>These findings highlight the need for bias-aware development and provide a foundation for building non-discriminatory deepfake detection systems.
  arXiv  Detail & Related papers  (2025-08-11T12:58:37Z)
• EMO-Debias: Benchmarking Gender Debiasing Techniques in Multi-Label Speech Emotion Recognition [49.27067541740956]
  EMO-Debias is a large-scale comparison of 13 debiasing methods applied to multi-label SER.<n>Our study encompasses techniques from pre-processing, regularization, adversarial learning, biased learners, and distributionally robust optimization.<n>Our analysis quantifies the trade-offs between fairness and accuracy, identifying which approaches consistently reduce gender performance gaps.
  arXiv  Detail & Related papers  (2025-06-05T05:48:31Z)
• FADEL: Uncertainty-aware Fake Audio Detection with Evidential Deep Learning [9.960675988638805]
  We propose a novel framework called fake audio detection with evidential learning (FADEL)<n>FADEL incorporates model uncertainty into its predictions, thereby leading to more robust performance in OOD scenarios.<n>We demonstrate the validity of uncertainty estimation by analyzing a strong correlation between average uncertainty and equal error rate (EER) across different spoofing algorithms.
  arXiv  Detail & Related papers  (2025-04-22T07:40:35Z)
• $C^2$AV-TSE: Context and Confidence-aware Audio Visual Target Speaker Extraction [80.57232374640911]
  We propose a model-agnostic strategy called the Mask-And-Recover (MAR)<n>MAR integrates both inter- and intra-modality contextual correlations to enable global inference within extraction modules.<n>To better target challenging parts within each sample, we introduce a Fine-grained Confidence Score (FCS) model.
  arXiv  Detail & Related papers  (2025-04-01T13:01:30Z)
• Exploring Gender Disparities in Automatic Speech Recognition Technology [22.729651340592586]
  We analyze how performance varies across different gender representations in training data.<n>Our findings suggest a complex interplay between the gender ratio in training data and ASR performance.
  arXiv  Detail & Related papers  (2025-02-25T18:29:38Z)
• Everyone deserves their voice to be heard: Analyzing Predictive Gender Bias in ASR Models Applied to Dutch Speech Data [13.91630413828167]
  This study focuses on identifying the performance disparities of Whisper models on Dutch speech data. We analyzed the word error rate, character error rate and a BERT-based semantic similarity across gender groups.
  arXiv  Detail & Related papers  (2024-11-14T13:29:09Z)
• Where are we in audio deepfake detection? A systematic analysis over generative and detection models [59.09338266364506]
  SONAR is a synthetic AI-Audio Detection Framework and Benchmark.<n>It provides a comprehensive evaluation for distinguishing cutting-edge AI-synthesized auditory content.<n>It is the first framework to uniformly benchmark AI-audio detection across both traditional and foundation model-based detection systems.
  arXiv  Detail & Related papers  (2024-10-06T01:03:42Z)
• No Pitch Left Behind: Addressing Gender Unbalance in Automatic Speech Recognition through Pitch Manipulation [20.731375136671605]
  We propose a data augmentation technique that manipulates the fundamental frequency (f0) and formants. This technique reduces the data unbalance among genders by simulating voices of the under-represented female speakers. Experiments on spontaneous English speech show that our technique yields a relative WER improvement up to 9.87% for utterances by female speakers.
  arXiv  Detail & Related papers  (2023-10-10T12:55:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.