Applying Wav2vec2.0 to Speech Recognition in Various Low-resource Languages

• URL: http://arxiv.org/abs/2012.12121v2
• Date: Sun, 17 Jan 2021 16:29:50 GMT
• Title: Applying Wav2vec2.0 to Speech Recognition in Various Low-resource Languages
• Authors: Cheng Yi, Jianzhong Wang, Ning Cheng, Shiyu Zhou, Bo Xu
• Abstract summary: In the speech domain, wav2vec2.0 starts to show its powerful representation ability and feasibility of ultra-low resource speech recognition on the Librispeech corpus. However, wav2vec2.0 has not been examined on real spoken scenarios and languages other than English. We apply pre-trained models to solve low-resource speech recognition tasks in various spoken languages.
• Score: 16.001329145018687
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: There are several domains that own corresponding widely used feature extractors, such as ResNet, BERT, and GPT-x. These models are usually pre-trained on large amounts of unlabeled data by self-supervision and can be effectively applied to downstream tasks. In the speech domain, wav2vec2.0 starts to show its powerful representation ability and feasibility of ultra-low resource speech recognition on the Librispeech corpus, which belongs to the audiobook domain. However, wav2vec2.0 has not been examined on real spoken scenarios and languages other than English. To verify its universality over languages, we apply pre-trained models to solve low-resource speech recognition tasks in various spoken languages. We achieve more than 20% relative improvements in six languages compared with previous work. Among these languages, English achieves a gain of 52.4%. Moreover, using coarse-grained modeling units, such as subword or character, achieves better results than fine-grained modeling units, such as phone or letter.

Related papers

• XLAVS-R: Cross-Lingual Audio-Visual Speech Representation Learning for Noise-Robust Speech Perception [62.660135152900615]
  Speech recognition and translation systems perform poorly on noisy inputs. XLAVS-R is a cross-lingual audio-visual speech representation model for noise-robust speech recognition and translation.
  arXiv  Detail & Related papers  (2024-03-21T13:52:17Z)
• Multilingual self-supervised speech representations improve the speech recognition of low-resource African languages with codeswitching [65.74653592668743]
  Finetuning self-supervised multilingual representations reduces absolute word error rates by up to 20%. In circumstances with limited training data finetuning self-supervised representations is a better performing and viable solution.
  arXiv  Detail & Related papers  (2023-11-25T17:05:21Z)
• AudioPaLM: A Large Language Model That Can Speak and Listen [79.44757696533709]
  We introduce AudioPaLM, a large language model for speech understanding and generation. AudioPaLM fuses text-based and speech-based language models. It can process and generate text and speech with applications including speech recognition and speech-to-speech translation.
  arXiv  Detail & Related papers  (2023-06-22T14:37:54Z)
• Scaling Speech Technology to 1,000+ Languages [66.31120979098483]
  The Massively Multilingual Speech (MMS) project increases the number of supported languages by 10-40x, depending on the task. Main ingredients are a new dataset based on readings of publicly available religious texts. We built pre-trained wav2vec 2.0 models covering 1,406 languages, a single multilingual automatic speech recognition model for 1,107 languages, speech synthesis models for the same number of languages, and a language identification model for 4,017 languages.
  arXiv  Detail & Related papers  (2023-05-22T22:09:41Z)
• Hindi as a Second Language: Improving Visually Grounded Speech with Semantically Similar Samples [89.16814518860357]
  The objective of this work is to explore the learning of visually grounded speech models (VGS) from multilingual perspective. Our key contribution in this work is to leverage the power of a high-resource language in a bilingual visually grounded speech model to improve the performance of a low-resource language.
  arXiv  Detail & Related papers  (2023-03-30T16:34:10Z)
• Multilingual Zero Resource Speech Recognition Base on Self-Supervise Pre-Trained Acoustic Models [14.887781621924255]
  This paper is the first attempt to extend the use of pre-trained models into word-level zero-resource speech recognition. It is done by fine-tuning the pre-trained models on IPA phoneme transcriptions and decoding with a language model trained on extra texts. Experiments on Wav2vec 2.0 and HuBERT models show that this method can achieve less than 20% word error rate on some languages.
  arXiv  Detail & Related papers  (2022-10-13T12:11:18Z)
• Brazilian Portuguese Speech Recognition Using Wav2vec 2.0 [0.26097841018267615]
  This work presents the development of a public Automatic Speech Recognition system using only open available audio data. The final model presents a Word Error Rate of 11.95% (Common Voice dataset) This corresponds to 13% less than the best open Automatic Speech Recognition model for Brazilian Portuguese available according to our best knowledge.
  arXiv  Detail & Related papers  (2021-07-23T18:54:39Z)
• Leveraging neural representations for facilitating access to untranscribed speech from endangered languages [10.61744395262441]
  We use data selected from 7 Australian Aboriginal languages and a regional variety of Dutch. We find that representations from the middle layers of the wav2vec 2.0 Transformer offer large gains in task performance. While features extracted using the pre-trained English model yielded improved detection on all the evaluation languages, better detection performance was associated with the evaluation language's phonological similarity to English.
  arXiv  Detail & Related papers  (2021-03-26T16:44:08Z)
• Unsupervised Cross-lingual Representation Learning for Speech Recognition [63.85924123692923]
  XLSR learns cross-lingual speech representations by pretraining a single model from the raw waveform of speech in multiple languages. We build on wav2vec 2.0 which is trained by solving a contrastive task over masked latent speech representations. Experiments show that cross-lingual pretraining significantly outperforms monolingual pretraining.
  arXiv  Detail & Related papers  (2020-06-24T18:25:05Z)
• Multilingual acoustic word embedding models for processing zero-resource languages [37.78342106714364]
  We train a single supervised embedding model on labelled data from multiple well-resourced languages. We then apply it to unseen zero-resource languages.
  arXiv  Detail & Related papers  (2020-02-06T05:53:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.