LearnerVoice: A Dataset of Non-Native English Learners' Spontaneous Speech

• URL: http://arxiv.org/abs/2407.04280v2
• Date: Fri, 4 Oct 2024 07:25:18 GMT
• Title: LearnerVoice: A Dataset of Non-Native English Learners' Spontaneous Speech
• Abstract summary: We publicly release LearnerVoice, a dataset consisting of 50.04 hours of audio and transcriptions of L2 learners' spontaneous speech. Our linguistic analysis reveals that transcriptions in our dataset contain L2S features, significantly more than native speech datasets. Fine-tuning whisper-small.en with LearnerVoice achieves a WER of 10.26%, 44.2% lower than vanilla whisper-small.en.
• Score: 36.447412884392115
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Prevalent ungrammatical expressions and disfluencies in spontaneous speech from second language (L2) learners pose unique challenges to Automatic Speech Recognition (ASR) systems. However, few datasets are tailored to L2 learner speech. We publicly release LearnerVoice, a dataset consisting of 50.04 hours of audio and transcriptions of L2 learners' spontaneous speech. Our linguistic analysis reveals that transcriptions in our dataset contain L2S (L2 learner's Spontaneous speech) features, consisting of ungrammatical expressions and disfluencies (e.g., filler words, word repetitions, self-repairs, false starts), significantly more than native speech datasets. Fine-tuning whisper-small.en with LearnerVoice achieves a WER of 10.26%, 44.2% lower than vanilla whisper-small.en. Furthermore, our qualitative analysis indicates that 54.2% of errors from the vanilla model on LearnerVoice are attributable to L2S features, with 48.1% of them being reduced in the fine-tuned model.

Related papers

• CosyVoice 3: Towards In-the-wild Speech Generation via Scaling-up and Post-training [70.31925012315064]
  We present CosyVoice 3, an improved model designed for zero-shot multilingual speech synthesis in the wild.<n>Key features of CosyVoice 3 include a novel speech tokenizer to improve prosody naturalness.<n>Data is expanded from ten thousand hours to one million hours, encompassing 9 languages and 18 Chinese dialects.
  arXiv  Detail & Related papers  (2025-05-23T07:55:21Z)
• S2S-Arena, Evaluating Speech2Speech Protocols on Instruction Following with Paralinguistic Information [47.950757976473035]
  We introduce S2S-Arena, a novel arena-style S2S benchmark that evaluates instruction-following capabilities with paralinguistic information. In addition to the superior performance of GPT-4o, the speech model of cascaded ASR, LLM, and TTS outperforms the jointly trained model after text-speech alignment in speech2speech protocols.
  arXiv  Detail & Related papers  (2025-03-07T02:07:00Z)
• Speak & Improve Corpus 2025: an L2 English Speech Corpus for Language Assessment and Feedback [28.53752312060031]
  Speak & Improve Corpus 2025 is a dataset of L2 learner English data with holistic scores and language error annotation. The aim of the corpus release is to address a major challenge to developing L2 spoken language processing systems. It is being made available for non-commercial use on the ELiT website.
  arXiv  Detail & Related papers  (2024-12-16T17:07:26Z)
• Multilingual self-supervised speech representations improve the speech recognition of low-resource African languages with codeswitching [65.74653592668743]
  Finetuning self-supervised multilingual representations reduces absolute word error rates by up to 20%. In circumstances with limited training data finetuning self-supervised representations is a better performing and viable solution.
  arXiv  Detail & Related papers  (2023-11-25T17:05:21Z)
• Voxtlm: unified decoder-only models for consolidating speech recognition/synthesis and speech/text continuation tasks [61.3055230762097]
  We propose a decoder-only language model, VoxtLM, that can perform four tasks: speech recognition, speech synthesis, text generation, and speech continuation. VoxtLM integrates text vocabulary with discrete speech tokens from self-supervised speech features and uses special tokens to enable multitask learning.
  arXiv  Detail & Related papers  (2023-09-14T03:13:18Z)
• Incorporating L2 Phonemes Using Articulatory Features for Robust Speech Recognition [2.8360662552057323]
  This study is on the efficient incorporation of the L2 phonemes, which in this work refer to Korean phonemes, through articulatory feature analysis. We employ the lattice-free maximum mutual information (LF-MMI) objective in an end-to-end manner, to train the acoustic model to align and predict one of multiple pronunciation candidates. Experimental results show that the proposed method improves ASR accuracy for Korean L2 speech by training solely on L1 speech data.
  arXiv  Detail & Related papers  (2023-06-05T01:55:33Z)
• Self-Supervised Speech Representations Preserve Speech Characteristics while Anonymizing Voices [15.136348385992047]
  We train several voice conversion models using self-supervised speech representations. Converted voices retain a low word error rate within 1% of the original voice. Experiments on dysarthric speech data show that speech features relevant to articulation, prosody, phonation and phonology can be extracted from anonymized voices.
  arXiv  Detail & Related papers  (2022-04-04T17:48:01Z)
• Wav2vec-Switch: Contrastive Learning from Original-noisy Speech Pairs for Robust Speech Recognition [52.71604809100364]
  We propose wav2vec-Switch, a method to encode noise robustness into contextualized representations of speech. Specifically, we feed original-noisy speech pairs simultaneously into the wav2vec 2.0 network. In addition to the existing contrastive learning task, we switch the quantized representations of the original and noisy speech as additional prediction targets.
  arXiv  Detail & Related papers  (2021-10-11T00:08:48Z)
• Continual-wav2vec2: an Application of Continual Learning for Self-Supervised Automatic Speech Recognition [0.23872611575805824]
  We present a method for continual learning of speech representations for multiple languages using self-supervised learning (SSL) Wav2vec models perform SSL on raw audio in a pretraining phase and then finetune on a small fraction of annotated data. We use ideas from continual learning to transfer knowledge from a previous task to speed up pretraining a new language task.
  arXiv  Detail & Related papers  (2021-07-26T10:39:03Z)
• Attention-based Contextual Language Model Adaptation for Speech Recognition [13.516224963932858]
  We introduce an attention mechanism for training neural speech recognition language models on text and non-linguistic contextual data. Our method reduces perplexity by 7.0% relative over a standard LM that does not incorporate contextual information.
  arXiv  Detail & Related papers  (2021-06-02T20:19:57Z)
• Data augmentation using prosody and false starts to recognize non-native children's speech [12.911954427107977]
  This paper describes AaltoASR's speech recognition system for the INTERSPEECH 2020 shared task on Automatic Speech Recognition. The task is to recognize non-native speech from children of various age groups given a limited amount of speech.
  arXiv  Detail & Related papers  (2020-08-29T05:32:32Z)
• LRSpeech: Extremely Low-Resource Speech Synthesis and Recognition [148.43282526983637]
  We develop LRSpeech, a TTS and ASR system for languages with low data cost. We conduct experiments on an experimental language (English) and a truly low-resource language (Lithuanian) to verify the effectiveness of LRSpeech. We are currently deploying LRSpeech into a commercialized cloud speech service to support TTS on more rare languages.
  arXiv  Detail & Related papers  (2020-08-09T08:16:33Z)
• Unsupervised Cross-lingual Representation Learning for Speech Recognition [63.85924123692923]
  XLSR learns cross-lingual speech representations by pretraining a single model from the raw waveform of speech in multiple languages. We build on wav2vec 2.0 which is trained by solving a contrastive task over masked latent speech representations. Experiments show that cross-lingual pretraining significantly outperforms monolingual pretraining.
  arXiv  Detail & Related papers  (2020-06-24T18:25:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.