Beyond the Labels: Unveiling Text-Dependency in Paralinguistic Speech Recognition Datasets

• URL: http://arxiv.org/abs/2403.07767v2
• Date: Fri, 18 Oct 2024 20:46:05 GMT
• Title: Beyond the Labels: Unveiling Text-Dependency in Paralinguistic Speech Recognition Datasets
• Authors: Jan Pešán, Santosh Kesiraju, Lukáš Burget, Jan ''Honza'' Černocký,
• Abstract summary: This paper critically evaluates the prevalent assumption that machine learning models genuinely learn to identify paralinguistic traits. By examining the lexical overlap in these datasets and testing the performance of machine learning models, we expose significant text-dependency in trait-labeling.
• Score: 0.5999777817331317
• License:
• Abstract: Paralinguistic traits like cognitive load and emotion are increasingly recognized as pivotal areas in speech recognition research, often examined through specialized datasets like CLSE and IEMOCAP. However, the integrity of these datasets is seldom scrutinized for text-dependency. This paper critically evaluates the prevalent assumption that machine learning models trained on such datasets genuinely learn to identify paralinguistic traits, rather than merely capturing lexical features. By examining the lexical overlap in these datasets and testing the performance of machine learning models, we expose significant text-dependency in trait-labeling. Our results suggest that some machine learning models, especially large pre-trained models like HuBERT, might inadvertently focus on lexical characteristics rather than the intended paralinguistic features. The study serves as a call to action for the research community to reevaluate the reliability of existing datasets and methodologies, ensuring that machine learning models genuinely learn what they are designed to recognize.

Related papers

• Capturing Pertinent Symbolic Features for Enhanced Content-Based Misinformation Detection [0.0]
  The detection of misleading content presents a significant hurdle due to its extreme linguistic and domain variability. This paper analyzes the linguistic attributes that characterize this phenomenon and how representative of such features some of the most popular misinformation datasets are. We demonstrate that the appropriate use of pertinent symbolic knowledge in combination with neural language models is helpful in detecting misleading content.
  arXiv  Detail & Related papers  (2024-01-29T16:42:34Z)
• Self-Supervised Learning for Audio-Based Emotion Recognition [1.7598252755538808]
  Self-supervised learning is a family of methods which can learn despite a scarcity of supervised labels. We have applied self-supervised learning pre-training to the classification of emotions from the CMU- MOSEI's acoustic modality. We find that self-supervised learning consistently improves the performance of the model across all metrics.
  arXiv  Detail & Related papers  (2023-07-23T14:40:50Z)
• Data AUDIT: Identifying Attribute Utility- and Detectability-Induced Bias in Task Models [8.420252576694583]
  We present a first technique for the rigorous, quantitative screening of medical image datasets. Our method decomposes the risks associated with dataset attributes in terms of their detectability and utility. Using our method, we show our screening method reliably identifies nearly imperceptible bias-inducing artifacts.
  arXiv  Detail & Related papers  (2023-04-06T16:50:15Z)
• An Empirical Investigation of Commonsense Self-Supervision with Knowledge Graphs [67.23285413610243]
  Self-supervision based on the information extracted from large knowledge graphs has been shown to improve the generalization of language models. We study the effect of knowledge sampling strategies and sizes that can be used to generate synthetic data for adapting language models.
  arXiv  Detail & Related papers  (2022-05-21T19:49:04Z)
• Predicting the Reproducibility of Social and Behavioral Science Papers Using Supervised Learning Models [21.69933721765681]
  We propose a framework that extracts five types of features from scholarly work that can be used to support assessments of published research claims. We analyze pairwise correlations between individual features and their importance for predicting a set of human-assessed ground truth labels.
  arXiv  Detail & Related papers  (2021-04-08T00:45:20Z)
• Prototypical Representation Learning for Relation Extraction [56.501332067073065]
  This paper aims to learn predictive, interpretable, and robust relation representations from distantly-labeled data. We learn prototypes for each relation from contextual information to best explore the intrinsic semantics of relations. Results on several relation learning tasks show that our model significantly outperforms the previous state-of-the-art relational models.
  arXiv  Detail & Related papers  (2021-03-22T08:11:43Z)
• Diverse Complexity Measures for Dataset Curation in Self-driving [80.55417232642124]
  We propose a new data selection method that exploits a diverse set of criteria that quantize interestingness of traffic scenes. Our experiments show that the proposed curation pipeline is able to select datasets that lead to better generalization and higher performance.
  arXiv  Detail & Related papers  (2021-01-16T23:45:02Z)
• Feature Learning for Accelerometer based Gait Recognition [0.0]
  Autoencoders are very close to discriminative end-to-end models with regards to their feature learning ability. fully convolutional models are able to learn good feature representations, regardless of the training strategy.
  arXiv  Detail & Related papers  (2020-07-31T10:58:01Z)
• Causal Feature Selection for Algorithmic Fairness [61.767399505764736]
  We consider fairness in the integration component of data management. We propose an approach to identify a sub-collection of features that ensure the fairness of the dataset.
  arXiv  Detail & Related papers  (2020-06-10T20:20:10Z)
• Temporal Embeddings and Transformer Models for Narrative Text Understanding [72.88083067388155]
  We present two approaches to narrative text understanding for character relationship modelling. The temporal evolution of these relations is described by dynamic word embeddings, that are designed to learn semantic changes over time. A supervised learning approach based on the state-of-the-art transformer model BERT is used instead to detect static relations between characters.
  arXiv  Detail & Related papers  (2020-03-19T14:23:12Z)
• ORB: An Open Reading Benchmark for Comprehensive Evaluation of Machine Reading Comprehension [53.037401638264235]
  We present an evaluation server, ORB, that reports performance on seven diverse reading comprehension datasets. The evaluation server places no restrictions on how models are trained, so it is a suitable test bed for exploring training paradigms and representation learning.
  arXiv  Detail & Related papers  (2019-12-29T07:27:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.