Towards Unsupervised Speech Recognition Without Pronunciation Models

• URL: http://arxiv.org/abs/2406.08380v1
• Date: Wed, 12 Jun 2024 16:30:58 GMT
• Title: Towards Unsupervised Speech Recognition Without Pronunciation Models
• Authors: Junrui Ni, Liming Wang, Yang Zhang, Kaizhi Qian, Heting Gao, Mark Hasegawa-Johnson, Chang D. Yoo,
• Abstract summary: Most languages lack sufficient paired speech and text data to effectively train automatic speech recognition systems. We propose the removal of reliance on a phoneme lexicon to develop unsupervised ASR systems. We experimentally demonstrate that an unsupervised speech recognizer can emerge from joint speech-to-speech and text-to-text masked token-infilling.
• Score: 57.222729245842054
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent advancements in supervised automatic speech recognition (ASR) have achieved remarkable performance, largely due to the growing availability of large transcribed speech corpora. However, most languages lack sufficient paired speech and text data to effectively train these systems. In this article, we tackle the challenge of developing ASR systems without paired speech and text corpora by proposing the removal of reliance on a phoneme lexicon. We explore a new research direction: word-level unsupervised ASR. Using a curated speech corpus containing only high-frequency English words, our system achieves a word error rate of nearly 20% without parallel transcripts or oracle word boundaries. Furthermore, we experimentally demonstrate that an unsupervised speech recognizer can emerge from joint speech-to-speech and text-to-text masked token-infilling. This innovative model surpasses the performance of previous unsupervised ASR models trained with direct distribution matching.

Related papers

• XLS-R fine-tuning on noisy word boundaries for unsupervised speech segmentation into words [13.783996617841467]
  We fine-tune an XLS-R model to predict word boundaries produced by top-tier speech segmentation systems. Our system can segment speech from languages unseen during fine-tuning in a zero-shot fashion.
  arXiv  Detail & Related papers  (2023-10-08T17:05:00Z)
• ContextSpeech: Expressive and Efficient Text-to-Speech for Paragraph Reading [65.88161811719353]
  This work develops a lightweight yet effective Text-to-Speech system, ContextSpeech. We first design a memory-cached recurrence mechanism to incorporate global text and speech context into sentence encoding. We construct hierarchically-structured textual semantics to broaden the scope for global context enhancement. Experiments show that ContextSpeech significantly improves the voice quality and prosody in paragraph reading with competitive model efficiency.
  arXiv  Detail & Related papers  (2023-07-03T06:55:03Z)
• Introducing Semantics into Speech Encoders [91.37001512418111]
  We propose an unsupervised way of incorporating semantic information from large language models into self-supervised speech encoders without labeled audio transcriptions. Our approach achieves similar performance as supervised methods trained on over 100 hours of labeled audio transcripts.
  arXiv  Detail & Related papers  (2022-11-15T18:44:28Z)
• Evaluating context-invariance in unsupervised speech representations [15.67794428589585]
  Current benchmarks do not measure context-invariance. We develop a new version of the ZeroSpeech ABX benchmark that measures context-invariance. We demonstrate that the context-independence of representations is predictive of the stability of word-level representations.
  arXiv  Detail & Related papers  (2022-10-27T21:15:49Z)
• Unsupervised Text-to-Speech Synthesis by Unsupervised Automatic Speech Recognition [60.84668086976436]
  An unsupervised text-to-speech synthesis (TTS) system learns to generate the speech waveform corresponding to any written sentence in a language. This paper proposes an unsupervised TTS system by leveraging recent advances in unsupervised automatic speech recognition (ASR) Our unsupervised system can achieve comparable performance to the supervised system in seven languages with about 10-20 hours of speech each.
  arXiv  Detail & Related papers  (2022-03-29T17:57:53Z)
• Comparing Supervised Models And Learned Speech Representations For Classifying Intelligibility Of Disordered Speech On Selected Phrases [11.3463024120429]
  We develop and compare different deep learning techniques to classify the intelligibility of disordered speech on selected phrases. We collected samples from a diverse set of 661 speakers with a variety of self-reported disorders speaking 29 words or phrases.
  arXiv  Detail & Related papers  (2021-07-08T17:24:25Z)
• Instant One-Shot Word-Learning for Context-Specific Neural Sequence-to-Sequence Speech Recognition [62.997667081978825]
  We present an end-to-end ASR system with a word/phrase memory and a mechanism to access this memory to recognize the words and phrases correctly. In this paper we demonstrate that through this mechanism our system is able to recognize more than 85% of newly added words that it previously failed to recognize.
  arXiv  Detail & Related papers  (2021-07-05T21:08:34Z)
• Unsupervised Automatic Speech Recognition: A Review [2.6212127510234797]
  We review the research literature to identify models and ideas that could lead to fully unsupervised ASR. The objective of the study is to identify the limitations of what can be learned from speech data alone and to understand the minimum requirements for speech recognition.
  arXiv  Detail & Related papers  (2021-06-09T08:33:20Z)
• Dynamic Acoustic Unit Augmentation With BPE-Dropout for Low-Resource End-to-End Speech Recognition [62.94773371761236]
  We consider building an effective end-to-end ASR system in low-resource setups with a high OOV rate. We propose a method of dynamic acoustic unit augmentation based on the BPE-dropout technique. Our monolingual Turkish Conformer established a competitive result with 22.2% character error rate (CER) and 38.9% word error rate (WER)
  arXiv  Detail & Related papers  (2021-03-12T10:10:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.