A Large Dataset of Spontaneous Speech with the Accent Spoken in São Paulo for Automatic Speech Recognition Evaluation

• URL: http://arxiv.org/abs/2409.15350v1
• Date: Tue, 10 Sep 2024 21:45:06 GMT
• Title: A Large Dataset of Spontaneous Speech with the Accent Spoken in São Paulo for Automatic Speech Recognition Evaluation
• Authors: Rodrigo Lima, Sidney Evaldo Leal, Arnaldo Candido Junior, Sandra Maria Aluísio,
• Abstract summary: We present a freely available spontaneous speech corpus for the Brazilian Portuguese language. We report preliminary automatic speech recognition (ASR) results, using both the Wav2Vec2-XLSR-53 and Distil-Whisper models. To the best of our knowledge, this is the first large Paulistano accented spontaneous speech corpus dedicated to the ASR task in Portuguese.
• Score: 0.8929537856623965
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: We present a freely available spontaneous speech corpus for the Brazilian Portuguese language and report preliminary automatic speech recognition (ASR) results, using both the Wav2Vec2-XLSR-53 and Distil-Whisper models fine-tuned and trained on our corpus. The NURC-SP Audio Corpus comprises 401 different speakers (204 females, 197 males) with a total of 239.30 hours of transcribed audio recordings. To the best of our knowledge, this is the first large Paulistano accented spontaneous speech corpus dedicated to the ASR task in Portuguese. We first present the design and development procedures of the NURC-SP Audio Corpus, and then describe four ASR experiments in detail. The experiments demonstrated promising results for the applicability of the corpus for ASR. Specifically, we fine-tuned two versions of Wav2Vec2-XLSR-53 model, trained a Distil-Whisper model using our dataset with labels determined by Whisper Large-V3 model, and fine-tuned this Distil-Whisper model with our corpus. Our best results were the Distil-Whisper fine-tuned over NURC-SP Audio Corpus with a WER of 24.22% followed by a fine-tuned versions of Wav2Vec2-XLSR-53 model with a WER of 33.73%, that is almost 10% point worse than Distil-Whisper's. To enable experiment reproducibility, we share the NURC-SP Audio Corpus dataset, pre-trained models, and training recipes in Hugging-Face and Github repositories.

Related papers

• GigaSpeech 2: An Evolving, Large-Scale and Multi-domain ASR Corpus for Low-Resource Languages with Automated Crawling, Transcription and Refinement [36.29371629234269]
  GigaSpeech 2 is a large-scale, multi-domain, multilingual speech recognition corpus. It comprises about 30,000 hours of automatically transcribed speech, including Thai, Indonesian, and Vietnamese.
  arXiv  Detail & Related papers  (2024-06-17T13:44:20Z)
• Prompting the Hidden Talent of Web-Scale Speech Models for Zero-Shot Task Generalization [61.60501633397704]
  We investigate the emergent abilities of the recently proposed web-scale speech model Whisper, by adapting it to unseen tasks with prompt engineering. We design task-specific prompts, by either leveraging another large-scale model, or simply manipulating the special tokens in the default prompts. Experiments show that our proposed prompts improve performance by 10% to 45% on the three zero-shot tasks, and even outperform SotA supervised models on some datasets.
  arXiv  Detail & Related papers  (2023-05-18T16:32:58Z)
• Hearing voices at the National Library -- a speech corpus and acoustic model for the Swedish language [0.0]
  We develop new acoustic models for automated speech recognition (ASR) at the National Library of Sweden (KB) We evaluate different approaches for a viable speech-to-text pipeline for audiovisual resources in Swedish, using the wav2vec 2.0 architecture. We conclude by highlighting the potential of such technology for cultural heritage institutions with vast collections of previously unlabelled audiovisual data.
  arXiv  Detail & Related papers  (2022-05-06T06:06:00Z)
• Wav2vec-Switch: Contrastive Learning from Original-noisy Speech Pairs for Robust Speech Recognition [52.71604809100364]
  We propose wav2vec-Switch, a method to encode noise robustness into contextualized representations of speech. Specifically, we feed original-noisy speech pairs simultaneously into the wav2vec 2.0 network. In addition to the existing contrastive learning task, we switch the quantized representations of the original and noisy speech as additional prediction targets.
  arXiv  Detail & Related papers  (2021-10-11T00:08:48Z)
• Scenario Aware Speech Recognition: Advancements for Apollo Fearless Steps & CHiME-4 Corpora [70.46867541361982]
  We consider a general non-semantic speech representation, which is trained with a self-supervised criteria based on triplet loss called TRILL. We observe +5.42% and +3.18% relative WER improvement for the development and evaluation sets of Fearless Steps.
  arXiv  Detail & Related papers  (2021-09-23T00:43:32Z)
• USC: An Open-Source Uzbek Speech Corpus and Initial Speech Recognition Experiments [3.8673738158945326]
  We present a freely available speech corpus for the Uzbek language. We report preliminary automatic speech recognition (ASR) results using both the deep neural network hidden Markov model (DNN-HMM) and end-to-end (E2E) architectures.
  arXiv  Detail & Related papers  (2021-07-30T03:39:39Z)
• Large-Scale Self- and Semi-Supervised Learning for Speech Translation [48.06478781295623]
  We explore both pretraining and self-training by using the large Libri-Light speech audio corpus and language modeling with CommonCrawl. Our experiments improve over the previous state of the art by 2.6 BLEU on average on all four considered CoVoST 2 language pairs.
  arXiv  Detail & Related papers  (2021-04-14T07:44:52Z)
• Talk, Don't Write: A Study of Direct Speech-Based Image Retrieval [13.40010612226968]
  Speech-based image retrieval has been studied as a proxy for joint representation learning. It is unclear how well speech-based retrieval can work in practice. We show our best speech-based models can match or exceed cascaded ASR-to-text encoding when speech is spontaneous, accented, or otherwise hard to automatically transcribe.
  arXiv  Detail & Related papers  (2021-04-05T13:11:40Z)
• Unsupervised Cross-lingual Representation Learning for Speech Recognition [63.85924123692923]
  XLSR learns cross-lingual speech representations by pretraining a single model from the raw waveform of speech in multiple languages. We build on wav2vec 2.0 which is trained by solving a contrastive task over masked latent speech representations. Experiments show that cross-lingual pretraining significantly outperforms monolingual pretraining.
  arXiv  Detail & Related papers  (2020-06-24T18:25:05Z)
• End-to-end Whispered Speech Recognition with Frequency-weighted Approaches and Pseudo Whisper Pre-training [130.56878980058966]
  We present several approaches for end-to-end (E2E) recognition of whispered speech. We achieve an overall relative reduction of 19.8% in PER and 44.4% in CER on a relatively small whispered TIMIT corpus. As long as we have a good E2E model pre-trained on normal or pseudo-whispered speech, a relatively small set of whispered speech may suffice to obtain a reasonably good E2E whispered speech recognizer.
  arXiv  Detail & Related papers  (2020-05-05T07:08:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.