What You Train Is What You Get: Gender Bias, Training Composition, and Post-Hoc Mitigation in Audio Deepfake Detection

• URL: http://arxiv.org/abs/2607.09891v1
• Date: Fri, 10 Jul 2026 18:29:24 GMT
• Title: What You Train Is What You Get: Gender Bias, Training Composition, and Post-Hoc Mitigation in Audio Deepfake Detection
• Abstract summary: We investigate gender bias in audio deepfake detection using the ASVspoof5 dataset.<n>We train attack-specific models on nine training sets with different gender compositions.<n>We find that training data composition strongly predicts bias direction, with the underrepresented gender performing worse at test time.
• Score: 1.8315988392353102
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Audio deepfake detection models determine whether speech is genuine or artificially generated, but high overall accuracy can mask substantial performance disparities across demographic groups. In this work, we investigate gender bias in audio deepfake detection using the ASVspoof5 dataset. We use ASVspoof5 under a controlled custom split designed to isolate gender-composition effects. We train attack-specific models on nine training sets with different gender compositions, ranging from female-only to male-only. We use a ResNet18 classifier with LogSpectrogram and WavLM-Base+ features, and we evaluated six post-hoc threshold calibration methods. Experimental results show that training data composition strongly predicts bias direction, with the underrepresented gender performing worse at test time. WavLM-Base+ features are shown to produce gender performance gaps 3.0 to 4.3 times larger than LogSpectrogram under identical training conditions, and balanced training is found to reduce LogSpectrogram bias but leave WavLM bias largely intact. Moreover, all six calibration strategies, including Oracle calibration with full test-set label access, leave the Equal Error Rate gap unchanged at 1.317 pp, confirming that threshold adjustment cannot correct underlying score distribution disparities. Overall, these findings suggest that gender fairness in audio deepfake detection must be addressed at training time, as post-hoc methods can only partially mitigate the resulting disparities

Related papers

• Towards Trustworthy Audio Deepfake Detection: A Systematic Framework for Diagnosing and Mitigating Gender Bias [2.000890150701116]
  We present the first diagnosis-first framework that identifies bias source before applying targeted mitigation.<n>Our diagnosis shows that bias does not stem from imbalanced training data but from acoustic representation differences.<n> Adjusting the decision threshold separately per gender reduces unfairness by 54% to 75% at no cost to detection accuracy.
  arXiv  Detail & Related papers  (2026-05-09T17:53:04Z)
• Gender Fairness in Audio Deepfake Detection: Performance and Disparity Analysis [2.000890150701116]
  We have attempted a thorough analysis of gender dependent performance and fairness in audio deepfake detection models.<n>Our results show that even when the overall EER difference between genders appears low, fairness-aware evaluation reveals disparities in error distribution.<n>This work highlights the importance of fairness-aware evaluation for developing a more equitable, robust, and trustworthy audio deepfake detection system.
  arXiv  Detail & Related papers  (2026-03-09T22:52:12Z)
• Exploring and Mitigating Gender Bias in Encoder-Based Transformer Models [0.0]
  This paper investigates gender bias in contextualized word embeddings, a crucial component of transformer-based models.<n>To quantify the degree of bias, we introduce a novel metric, MALoR, which assesses bias based on model probabilities for filling masked tokens.<n>Our experiments reveal significant reductions in gender bias scores across different pronoun pairs.
  arXiv  Detail & Related papers  (2025-11-01T11:49:44Z)
• Bias in Gender Bias Benchmarks: How Spurious Features Distort Evaluation [116.86965910589775]
  We show that even minimal perturbations, such as masking just 10% of objects or weakly blurring backgrounds, can dramatically alter bias scores.<n>This suggests that current bias evaluations reflect model responses to spurious features rather than gender bias.
  arXiv  Detail & Related papers  (2025-09-09T11:14:11Z)
• AI Gender Bias, Disparities, and Fairness: Does Training Data Matter? [3.509963616428399]
  This study delves into the pervasive issue of gender issues in artificial intelligence (AI)<n>It analyzes more than 1000 human-graded student responses from male and female participants across six assessment items.<n>Results indicate that scoring accuracy for mixed-trained models shows an insignificant difference from either male- or female-trained models.
  arXiv  Detail & Related papers  (2023-12-17T22:37:06Z)
• The Impact of Debiasing on the Performance of Language Models in Downstream Tasks is Underestimated [70.23064111640132]
  We compare the impact of debiasing on performance across multiple downstream tasks using a wide-range of benchmark datasets. Experiments show that the effects of debiasing are consistently emphunderestimated across all tasks.
  arXiv  Detail & Related papers  (2023-09-16T20:25:34Z)
• Bridging Precision and Confidence: A Train-Time Loss for Calibrating Object Detection [58.789823426981044]
  We propose a novel auxiliary loss formulation that aims to align the class confidence of bounding boxes with the accurateness of predictions. Our results reveal that our train-time loss surpasses strong calibration baselines in reducing calibration error for both in and out-domain scenarios.
  arXiv  Detail & Related papers  (2023-03-25T08:56:21Z)
• Improving Gender Fairness of Pre-Trained Language Models without Catastrophic Forgetting [88.83117372793737]
  Forgetting information in the original training data may damage the model's downstream performance by a large margin. We propose GEnder Equality Prompt (GEEP) to improve gender fairness of pre-trained models with less forgetting.
  arXiv  Detail & Related papers  (2021-10-11T15:52:16Z)
• Balancing Biases and Preserving Privacy on Balanced Faces in the Wild [50.915684171879036]
  There are demographic biases present in current facial recognition (FR) models. We introduce our Balanced Faces in the Wild dataset to measure these biases across different ethnic and gender subgroups. We find that relying on a single score threshold to differentiate between genuine and imposters sample pairs leads to suboptimal results. We propose a novel domain adaptation learning scheme that uses facial features extracted from state-of-the-art neural networks.
  arXiv  Detail & Related papers  (2021-03-16T15:05:49Z)
• Mitigating Gender Bias in Captioning Systems [56.25457065032423]
  Most captioning models learn gender bias, leading to high gender prediction errors, especially for women. We propose a new Guided Attention Image Captioning model (GAIC) which provides self-guidance on visual attention to encourage the model to capture correct gender visual evidence.
  arXiv  Detail & Related papers  (2020-06-15T12:16:19Z)
• How Does Gender Balance In Training Data Affect Face Recognition Accuracy? [12.362029427868206]
  It is often speculated that lower accuracy for women is caused by under-representation in the training data. This work investigates female under-representation in the training data is truly the cause of lower accuracy for females on test data.
  arXiv  Detail & Related papers  (2020-02-07T18:11:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.