Curriculum optimization for low-resource speech recognition

• URL: http://arxiv.org/abs/2202.08883v1
• Date: Thu, 17 Feb 2022 19:47:50 GMT
• Title: Curriculum optimization for low-resource speech recognition
• Authors: Anastasia Kuznetsova, Anurag Kumar, Jennifer Drexler Fox, Francis Tyers
• Abstract summary: We propose an automated curriculum learning approach to optimize the sequence of training examples. We introduce a new difficulty measure called compression ratio that can be used as a scoring function for raw audio in various noise conditions.
• Score: 4.803994937990389
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Modern end-to-end speech recognition models show astonishing results in transcribing audio signals into written text. However, conventional data feeding pipelines may be sub-optimal for low-resource speech recognition, which still remains a challenging task. We propose an automated curriculum learning approach to optimize the sequence of training examples based on both the progress of the model while training and prior knowledge about the difficulty of the training examples. We introduce a new difficulty measure called compression ratio that can be used as a scoring function for raw audio in various noise conditions. The proposed method improves speech recognition Word Error Rate performance by up to 33% relative over the baseline system

Related papers

• Continuous Modeling of the Denoising Process for Speech Enhancement Based on Deep Learning [61.787485727134424]
  We use a state variable to indicate the denoising process. A UNet-like neural network learns to estimate every state variable sampled from the continuous denoising process. Experimental results indicate that preserving a small amount of noise in the clean target benefits speech enhancement.
  arXiv  Detail & Related papers  (2023-09-17T13:27:11Z)
• Improving Audio-Visual Speech Recognition by Lip-Subword Correlation Based Visual Pre-training and Cross-Modal Fusion Encoder [58.523884148942166]
  We propose two novel techniques to improve audio-visual speech recognition (AVSR) under a pre-training and fine-tuning training framework. First, we explore the correlation between lip shapes and syllable-level subword units in Mandarin to establish good frame-level syllable boundaries from lip shapes. Next, we propose an audio-guided cross-modal fusion encoder (CMFE) neural network to utilize main training parameters for multiple cross-modal attention layers.
  arXiv  Detail & Related papers  (2023-08-14T08:19:24Z)
• Improving the Intent Classification accuracy in Noisy Environment [9.447108578893639]
  In this paper, we investigate how environmental noise and related noise reduction techniques to address the intent classification task with end-to-end neural models. For this task, the use of speech enhancement greatly improves the classification accuracy in noisy conditions.
  arXiv  Detail & Related papers  (2023-03-12T06:11:44Z)
• Contextual-Utterance Training for Automatic Speech Recognition [65.4571135368178]
  We propose a contextual-utterance training technique which makes use of the previous and future contextual utterances. Also, we propose a dual-mode contextual-utterance training technique for streaming automatic speech recognition (ASR) systems. The proposed technique is able to reduce both the WER and the average last token emission latency by more than 6% and 40ms relative.
  arXiv  Detail & Related papers  (2022-10-27T08:10:44Z)
• Instance Regularization for Discriminative Language Model Pre-training [108.41891836796366]
  This work proposes to estimate the complexity of restoring the original sentences from corrupted ones in language model pre-training. Experimental results on natural language understanding and reading comprehension benchmarks show that our approach improves pre-training efficiency, effectiveness, and robustness.
  arXiv  Detail & Related papers  (2022-10-11T14:16:37Z)
• Lip-Listening: Mixing Senses to Understand Lips using Cross Modality Knowledge Distillation for Word-Based Models [0.03499870393443267]
  This work builds on recent state-of-the-art word-based lipreading models by integrating sequence-level and frame-level Knowledge Distillation (KD) to their systems. We propose a technique to transfer speech recognition capabilities from audio speech recognition systems to visual speech recognizers, where our goal is to utilize audio data during lipreading model training.
  arXiv  Detail & Related papers  (2022-06-05T15:47:54Z)
• On monoaural speech enhancement for automatic recognition of real noisy speech using mixture invariant training [33.79711018198589]
  We extend the existing mixture invariant training criterion to exploit both unpaired clean speech and real noisy data. It is found that the unpaired clean speech is crucial to improve quality of separated speech from real noisy speech. The proposed method also performs remixing of processed and unprocessed signals to alleviate the processing artifacts.
  arXiv  Detail & Related papers  (2022-05-03T19:37:58Z)
• Wav2Seq: Pre-training Speech-to-Text Encoder-Decoder Models Using Pseudo Languages [58.43299730989809]
  We introduce Wav2Seq, the first self-supervised approach to pre-train both parts of encoder-decoder models for speech data. We induce a pseudo language as a compact discrete representation, and formulate a self-supervised pseudo speech recognition task. This process stands on its own, or can be applied as low-cost second-stage pre-training.
  arXiv  Detail & Related papers  (2022-05-02T17:59:02Z)
• Improving Noise Robustness of Contrastive Speech Representation Learning with Speech Reconstruction [109.44933866397123]
  Noise robustness is essential for deploying automatic speech recognition systems in real-world environments. We employ a noise-robust representation learned by a refined self-supervised framework for noisy speech recognition. We achieve comparable performance to the best supervised approach reported with only 16% of labeled data.
  arXiv  Detail & Related papers  (2021-10-28T20:39:02Z)
• Adversarial Feature Learning and Unsupervised Clustering based Speech Synthesis for Found Data with Acoustic and Textual Noise [18.135965605011105]
  Attention-based sequence-to-sequence (seq2seq) speech synthesis has achieved extraordinary performance. A studio-quality corpus with manual transcription is necessary to train such seq2seq systems. We propose an approach to build high-quality and stable seq2seq based speech synthesis system using challenging found data.
  arXiv  Detail & Related papers  (2020-04-28T15:32:45Z)
• Contextual Text Denoising with Masked Language Models [21.923035129334373]
  We propose a new contextual text denoising algorithm based on the ready-to-use masked language model. The proposed algorithm does not require retraining of the model and can be integrated into any NLP system.
  arXiv  Detail & Related papers  (2019-10-30T18:47:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.