Evaluation of state-of-the-art ASR Models in Child-Adult Interactions

• URL: http://arxiv.org/abs/2409.16135v1
• Date: Tue, 24 Sep 2024 14:42:37 GMT
• Title: Evaluation of state-of-the-art ASR Models in Child-Adult Interactions
• Authors: Aditya Ashvin, Rimita Lahiri, Aditya Kommineni, Somer Bishop, Catherine Lord, Sudarsana Reddy Kadiri, Shrikanth Narayanan,
• Abstract summary: Speech foundation models show a noticeable performance drop (15-20% absolute WER) for child speech compared to adult speech in the conversational setting. We employ LoRA on the best performing zero shot model (whisper-large) to probe the effectiveness of fine-tuning in a low resource setting.
• Score: 27.30130353688078
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: The ability to reliably transcribe child-adult conversations in a clinical setting is valuable for diagnosis and understanding of numerous developmental disorders such as Autism Spectrum Disorder. Recent advances in deep learning architectures and availability of large scale transcribed data has led to development of speech foundation models that have shown dramatic improvements in ASR performance. However, the ability of these models to translate well to conversational child-adult interactions is under studied. In this work, we provide a comprehensive evaluation of ASR performance on a dataset containing child-adult interactions from autism diagnostic sessions, using Whisper, Wav2Vec2, HuBERT, and WavLM. We find that speech foundation models show a noticeable performance drop (15-20% absolute WER) for child speech compared to adult speech in the conversational setting. Then, we employ LoRA on the best performing zero shot model (whisper-large) to probe the effectiveness of fine-tuning in a low resource setting, resulting in ~8% absolute WER improvement for child speech and ~13% absolute WER improvement for adult speech.

Related papers

• Improving child speech recognition with augmented child-like speech [20.709414063132627]
  Cross-lingual child-to-child voice conversion significantly improved child ASR performance. State-of-the-art ASRs show suboptimal performance for child speech.
  arXiv  Detail & Related papers  (2024-06-12T08:56:46Z)
• Hear Me, See Me, Understand Me: Audio-Visual Autism Behavior Recognition [47.550391816383794]
  We introduce a novel problem of audio-visual autism behavior recognition. Social behavior recognition is an essential aspect previously omitted in AI-assisted autism screening research. We will release our dataset, code, and pre-trained models.
  arXiv  Detail & Related papers  (2024-03-22T22:52:35Z)
• A comparative analysis between Conformer-Transducer, Whisper, and wav2vec2 for improving the child speech recognition [2.965450563218781]
  We show that finetuning Conformer-transducer models on child speech yields significant improvements in ASR performance on child speech. We also show Whisper and wav2vec2 adaptation on different child speech datasets.
  arXiv  Detail & Related papers  (2023-11-07T19:32:48Z)
• Kid-Whisper: Towards Bridging the Performance Gap in Automatic Speech Recognition for Children VS. Adults [4.765434968114876]
  We enhance the utility of the MyST dataset through more efficient data preprocessing. We show that this improvement can be generalized to unseen datasets. Results showcase the viable and efficient integration of Whisper for effective children's speech recognition.
  arXiv  Detail & Related papers  (2023-09-12T06:58:18Z)
• Adaptation of Whisper models to child speech recognition [3.2548794659022398]
  We show that finetuning Whisper on child speech yields significant improvements in ASR performance on child speech. utilizing self-supervised Wav2vec2 models that have been finetuned on child speech outperforms Whisper finetuning.
  arXiv  Detail & Related papers  (2023-07-24T12:54:45Z)
• Analysing the Impact of Audio Quality on the Use of Naturalistic Long-Form Recordings for Infant-Directed Speech Research [62.997667081978825]
  Modelling of early language acquisition aims to understand how infants bootstrap their language skills. Recent developments have enabled the use of more naturalistic training data for computational models. It is currently unclear how the sound quality could affect analyses and modelling experiments conducted on such data.
  arXiv  Detail & Related papers  (2023-05-03T08:25:37Z)
• Leveraging Pretrained Representations with Task-related Keywords for Alzheimer's Disease Detection [69.53626024091076]
  Alzheimer's disease (AD) is particularly prominent in older adults. Recent advances in pre-trained models motivate AD detection modeling to shift from low-level features to high-level representations. This paper presents several efficient methods to extract better AD-related cues from high-level acoustic and linguistic features.
  arXiv  Detail & Related papers  (2023-03-14T16:03:28Z)
• Improving Children's Speech Recognition by Fine-tuning Self-supervised Adult Speech Representations [2.2191297646252646]
  Children's speech recognition is a vital, yet largely overlooked domain when building inclusive speech technologies. Recent advances in self-supervised learning have created a new opportunity for overcoming this problem of data scarcity. We leverage self-supervised adult speech representations and use three well-known child speech corpora to build models for children's speech recognition.
  arXiv  Detail & Related papers  (2022-11-14T22:03:36Z)
• Speaker Identity Preservation in Dysarthric Speech Reconstruction by Adversarial Speaker Adaptation [59.41186714127256]
  Dysarthric speech reconstruction (DSR) aims to improve the quality of dysarthric speech. Speaker encoder (SE) optimized for speaker verification has been explored to control the speaker identity. We propose a novel multi-task learning strategy, i.e., adversarial speaker adaptation (ASA)
  arXiv  Detail & Related papers  (2022-02-18T08:59:36Z)
• Investigation of Data Augmentation Techniques for Disordered Speech Recognition [69.50670302435174]
  This paper investigates a set of data augmentation techniques for disordered speech recognition. Both normal and disordered speech were exploited in the augmentation process. The final speaker adapted system constructed using the UASpeech corpus and the best augmentation approach based on speed perturbation produced up to 2.92% absolute word error rate (WER)
  arXiv  Detail & Related papers  (2022-01-14T17:09:22Z)
• A Preliminary Study of a Two-Stage Paradigm for Preserving Speaker Identity in Dysarthric Voice Conversion [50.040466658605524]
  We propose a new paradigm for maintaining speaker identity in dysarthric voice conversion (DVC) The poor quality of dysarthric speech can be greatly improved by statistical VC. But as the normal speech utterances of a dysarthria patient are nearly impossible to collect, previous work failed to recover the individuality of the patient.
  arXiv  Detail & Related papers  (2021-06-02T18:41:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.