A Toolkit for Detecting Spurious Correlations in Speech Datasets

• URL: http://arxiv.org/abs/2604.26676v1
• Date: Wed, 29 Apr 2026 13:47:22 GMT
• Title: A Toolkit for Detecting Spurious Correlations in Speech Datasets
• Abstract summary: Spurious correlations may arise due to heterogeneous recording conditions.<n>When present both in the training and test data, these correlations result in an overestimation of the system performance.<n>Our toolkit implements a diagnostic method based on the detection of the target class using only the non-speech regions in the audio.
• Score: 6.206128767866976
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: We introduce a toolkit for uncovering spurious correlations between recording characteristics and target class in speech datasets. Spurious correlations may arise due to heterogeneous recording conditions, a common scenario for health-related datasets. When present both in the training and test data, these correlations result in an overestimation of the system performance -- a dangerous situation, specially in high-stakes application where systems are required to satisfy minimum performance requirements. Our toolkit implements a diagnostic method based on the detection of the target class using only the non-speech regions in the audio. Better than chance performance at this task indicates that information about the target class can be extracted from the non-speech regions, flagging the presence of spurious correlations. The toolkit is publicly available for research use.

Related papers

• An Intervention-Based Framework for Shortcut Diagnosis in Spoofing Countermeasures [5.593982292458519]
  We propose an intervention-based diagnostic framework to identify confound-driven shortcut dependencies.<n>We operationalise this through controlled acoustic perturbations targeting non-speech structure, spectral content, and signal energy.<n>Results reveal that non-speech interventions produce the largest performance shifts, confirming non-speech intervals as a dominant shortcut.
  arXiv  Detail & Related papers  (2026-07-03T09:42:31Z)
• SpurAudio: A Benchmark for Studying Shortcut Learning in Few-Shot Audio Classification [4.791940743080381]
  Few-shot classification (FSC) is widely used for learning from limited labeled data, yet most evaluations implicitly assume that target concepts are independent of contextual cues.<n>In real-world settings, examples often appear within rich contexts, allowing models to exploit spurious correlations between foreground content and background signals.<n>We introduce SpurAudio, a benchmark that leverages the natural separability of foreground events and background environments in audio to enable controlled, multi-level evaluation of contextual shifts across support and query sets.
  arXiv  Detail & Related papers  (2026-05-13T15:32:57Z)
• Exploring the Frontiers of kNN Noisy Feature Detection and Recovery for Self-Driving Labs [0.49478969093606673]
  This study develops an automated workflow to detect noisy features, determine sample-feature pairings that can be corrected, and finally recover the correct feature values.<n>A systematic study is then performed to examine how dataset size, noise intensity, and feature value distribution affect both the detectability and recoverability of noisy features.
  arXiv  Detail & Related papers  (2025-07-15T03:35:56Z)
• A Hybrid Framework for Statistical Feature Selection and Image-Based Noise-Defect Detection [55.2480439325792]
  This paper presents a hybrid framework that integrates both statistical feature selection and classification techniques to improve defect detection accuracy.<n>We present around 55 distinguished features that are extracted from industrial images, which are then analyzed using statistical methods.<n>By integrating these methods with flexible machine learning applications, the proposed framework improves detection accuracy and reduces false positives and misclassifications.
  arXiv  Detail & Related papers  (2024-12-11T22:12:21Z)
• Towards Open-Vocabulary Audio-Visual Event Localization [59.23161248808759]
  We introduce the Open-Vocabulary Audio-Visual Event localization problem.<n>This problem requires localizing audio-visual events and predicting explicit categories for both seen and unseen data at inference.<n>We propose the OV-AVEBench dataset, comprising 24,800 videos across 67 real-life audio-visual scenes.
  arXiv  Detail & Related papers  (2024-11-18T04:35:20Z)
• Spuriousness-Aware Meta-Learning for Learning Robust Classifiers [26.544938760265136]
  Spurious correlations are brittle associations between certain attributes of inputs and target variables. Deep image classifiers often leverage them for predictions, leading to poor generalization on the data where the correlations do not hold. Mitigating the impact of spurious correlations is crucial towards robust model generalization, but it often requires annotations of the spurious correlations in data.
  arXiv  Detail & Related papers  (2024-06-15T21:41:25Z)
• Surgical Phase and Instrument Recognition: How to identify appropriate Dataset Splits [2.045596350476764]
  This work presents a publicly available data visualization tool that enables interactive exploration of dataset splits. It focuses on the visualization of the occurrence of phases, phase transitions, instruments, and instrument combinations across sets. Results: We performed an analysis of common Cholec80 dataset splits and were able to uncover phase transitions and combinations of instruments that were not represented in one of the sets.
  arXiv  Detail & Related papers  (2023-06-29T12:02:16Z)
• On the Universal Adversarial Perturbations for Efficient Data-free Adversarial Detection [55.73320979733527]
  We propose a data-agnostic adversarial detection framework, which induces different responses between normal and adversarial samples to UAPs. Experimental results show that our method achieves competitive detection performance on various text classification tasks.
  arXiv  Detail & Related papers  (2023-06-27T02:54:07Z)
• Vision-Language Modelling For Radiological Imaging and Reports In The Low Data Regime [70.04389979779195]
  This paper explores training medical vision-language models (VLMs) where the visual and language inputs are embedded into a common space. We explore several candidate methods to improve low-data performance, including adapting generic pre-trained models to novel image and text domains. Using text-to-image retrieval as a benchmark, we evaluate the performance of these methods with variable sized training datasets of paired chest X-rays and radiological reports.
  arXiv  Detail & Related papers  (2023-03-30T18:20:00Z)
• Speech Detection For Child-Clinician Conversations In Danish For Low-Resource In-The-Wild Conditions: A Case Study [6.4461798613033405]
  We study the performance of a pre-trained speech model on a dataset comprising of child-clinician conversations in Danish. We learned that the model with default classification threshold performs worse on children from the patient group. Our study on few-instance adaptation shows that three-minutes of clinician-child conversation is sufficient to obtain the optimum classification threshold.
  arXiv  Detail & Related papers  (2022-04-25T10:51:54Z)
• Deep Learning for Hate Speech Detection: A Comparative Study [54.42226495344908]
  We present here a large-scale empirical comparison of deep and shallow hate-speech detection methods. Our goal is to illuminate progress in the area, and identify strengths and weaknesses in the current state-of-the-art. In doing so we aim to provide guidance as to the use of hate-speech detection in practice, quantify the state-of-the-art, and identify future research directions.
  arXiv  Detail & Related papers  (2022-02-19T03:48:20Z)
• Unsupervised Domain Adaptation for Acoustic Scene Classification Using Band-Wise Statistics Matching [69.24460241328521]
  Machine learning algorithms can be negatively affected by mismatches between training (source) and test (target) data distributions. We propose an unsupervised domain adaptation method that consists of aligning the first- and second-order sample statistics of each frequency band of target-domain acoustic scenes to the ones of the source-domain training dataset. We show that the proposed method outperforms the state-of-the-art unsupervised methods found in the literature in terms of both source- and target-domain classification accuracy.
  arXiv  Detail & Related papers  (2020-04-30T23:56:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.