Reading Miscue Detection in Primary School through Automatic Speech Recognition

• URL: http://arxiv.org/abs/2406.07060v1
• Date: Tue, 11 Jun 2024 08:41:21 GMT
• Title: Reading Miscue Detection in Primary School through Automatic Speech Recognition
• Authors: Lingyun Gao, Cristian Tejedor-Garcia, Helmer Strik, Catia Cucchiarini,
• Abstract summary: This study investigates how efficiently state-of-the-art (SOTA) pretrained ASR models recognize Dutch native children speech. We found that Hubert Large finetuned on Dutch speech achieves SOTA phoneme-level child speech recognition. Wav2Vec2 Large shows the highest recall at 0.83, whereas Whisper exhibits the highest precision at 0.52 and an F1 score of 0.52.
• Score: 10.137389745562512
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Automatic reading diagnosis systems can benefit both teachers for more efficient scoring of reading exercises and students for accessing reading exercises with feedback more easily. However, there are limited studies on Automatic Speech Recognition (ASR) for child speech in languages other than English, and limited research on ASR-based reading diagnosis systems. This study investigates how efficiently state-of-the-art (SOTA) pretrained ASR models recognize Dutch native children speech and manage to detect reading miscues. We found that Hubert Large finetuned on Dutch speech achieves SOTA phoneme-level child speech recognition (PER at 23.1\%), while Whisper (Faster Whisper Large-v2) achieves SOTA word-level performance (WER at 9.8\%). Our findings suggest that Wav2Vec2 Large and Whisper are the two best ASR models for reading miscue detection. Specifically, Wav2Vec2 Large shows the highest recall at 0.83, whereas Whisper exhibits the highest precision at 0.52 and an F1 score of 0.52.

Related papers

• Automatic Speech Recognition of Non-Native Child Speech for Language Learning Applications [18.849741353784328]
  We assess the performance of two state-of-the-art ASR systems, Wav2Vec2.0 and Whisper AI. We evaluate their performance on read and extemporaneous speech of native and non-native Dutch children.
  arXiv  Detail & Related papers  (2023-06-29T06:14:26Z)
• From English to More Languages: Parameter-Efficient Model Reprogramming for Cross-Lingual Speech Recognition [50.93943755401025]
  We propose a new parameter-efficient learning framework based on neural model reprogramming for cross-lingual speech recognition. We design different auxiliary neural architectures focusing on learnable pre-trained feature enhancement. Our methods outperform existing ASR tuning architectures and their extension with self-supervised losses.
  arXiv  Detail & Related papers  (2023-01-19T02:37:56Z)
• Nonwords Pronunciation Classification in Language Development Tests for Preschool Children [7.224391516694955]
  This work aims to automatically evaluate whether the language development of children is age-appropriate. In this work, the task is to determine whether spoken nonwords have been uttered correctly. We compare different approaches that are motivated to model specific language structures.
  arXiv  Detail & Related papers  (2022-06-16T10:19:47Z)
• Cross-lingual Self-Supervised Speech Representations for Improved Dysarthric Speech Recognition [15.136348385992047]
  This study explores the usefulness of using Wav2Vec self-supervised speech representations as features for training an ASR system for dysarthric speech. We train an acoustic model with features extracted from Wav2Vec, Hubert, and the cross-lingual XLSR model. Results suggest that speech representations pretrained on large unlabelled data can improve word error rate (WER) performance.
  arXiv  Detail & Related papers  (2022-04-04T17:36:01Z)
• Automatic Speech recognition for Speech Assessment of Preschool Children [4.554894288663752]
  The acoustic and linguistic features of preschool speech are investigated in this study. Wav2Vec 2.0 is a paradigm that could be used to build a robust end-to-end speech recognition system.
  arXiv  Detail & Related papers  (2022-03-24T07:15:24Z)
• Sequence-level self-learning with multiple hypotheses [53.04725240411895]
  We develop new self-learning techniques with an attention-based sequence-to-sequence (seq2seq) model for automatic speech recognition (ASR) In contrast to conventional unsupervised learning approaches, we adopt the emphmulti-task learning (MTL) framework. Our experiment results show that our method can reduce the WER on the British speech data from 14.55% to 10.36% compared to the baseline model trained with the US English data only.
  arXiv  Detail & Related papers  (2021-12-10T20:47:58Z)
• A study on native American English speech recognition by Indian listeners with varying word familiarity level [62.14295630922855]
  We have three kinds of responses from each listener while they recognize an utterance. From these transcriptions, word error rate (WER) is calculated and used as a metric to evaluate the similarity between the recognized and the original sentences. Speaker nativity wise analysis shows that utterances from speakers of some nativity are more difficult to recognize by Indian listeners compared to few other nativities.
  arXiv  Detail & Related papers  (2021-12-08T07:43:38Z)
• Do We Still Need Automatic Speech Recognition for Spoken Language Understanding? [14.575551366682872]
  We show that learned speech features are superior to ASR transcripts on three classification tasks. We highlight the intrinsic robustness of wav2vec 2.0 representations to out-of-vocabulary words as key to better performance.
  arXiv  Detail & Related papers  (2021-11-29T15:13:36Z)
• Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition [97.44056170380726]
  We employ a combination of recent developments in semi-supervised learning for automatic speech recognition to obtain state-of-the-art results on LibriSpeech. We carry out noisy student training with SpecAugment using giant Conformer models pre-trained using wav2vec 2.0 pre-training. We are able to achieve word-error-rates (WERs) 1.4%/2.6% on the LibriSpeech test/test-other sets against the current state-of-the-art WERs 1.7%/3.3%.
  arXiv  Detail & Related papers  (2020-10-20T17:58:13Z)
• LRSpeech: Extremely Low-Resource Speech Synthesis and Recognition [148.43282526983637]
  We develop LRSpeech, a TTS and ASR system for languages with low data cost. We conduct experiments on an experimental language (English) and a truly low-resource language (Lithuanian) to verify the effectiveness of LRSpeech. We are currently deploying LRSpeech into a commercialized cloud speech service to support TTS on more rare languages.
  arXiv  Detail & Related papers  (2020-08-09T08:16:33Z)
• Unsupervised Cross-lingual Representation Learning for Speech Recognition [63.85924123692923]
  XLSR learns cross-lingual speech representations by pretraining a single model from the raw waveform of speech in multiple languages. We build on wav2vec 2.0 which is trained by solving a contrastive task over masked latent speech representations. Experiments show that cross-lingual pretraining significantly outperforms monolingual pretraining.
  arXiv  Detail & Related papers  (2020-06-24T18:25:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.