Multi-view Frequency LSTM: An Efficient Frontend for Automatic Speech Recognition

• URL: http://arxiv.org/abs/2007.00131v1
• Date: Tue, 30 Jun 2020 22:19:53 GMT
• Title: Multi-view Frequency LSTM: An Efficient Frontend for Automatic Speech Recognition
• Authors: Maarten Van Segbroeck, Harish Mallidih, Brian King, I-Fan Chen, Gurpreet Chadha, Roland Maas
• Abstract summary: We present a simple and efficient modification by combining the outputs of multiple FLSTM stacks with different views. We show that the multi-view FLSTM acoustic model provides relative Word Error Rate (WER) improvements of 3-7% for different speaker and acoustic environment scenarios.
• Score: 4.753402561130792
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Acoustic models in real-time speech recognition systems typically stack multiple unidirectional LSTM layers to process the acoustic frames over time. Performance improvements over vanilla LSTM architectures have been reported by prepending a stack of frequency-LSTM (FLSTM) layers to the time LSTM. These FLSTM layers can learn a more robust input feature to the time LSTM layers by modeling time-frequency correlations in the acoustic input signals. A drawback of FLSTM based architectures however is that they operate at a predefined, and tuned, window size and stride, referred to as 'view' in this paper. We present a simple and efficient modification by combining the outputs of multiple FLSTM stacks with different views, into a dimensionality reduced feature representation. The proposed multi-view FLSTM architecture allows to model a wider range of time-frequency correlations compared to an FLSTM model with single view. When trained on 50K hours of English far-field speech data with CTC loss followed by sMBR sequence training, we show that the multi-view FLSTM acoustic model provides relative Word Error Rate (WER) improvements of 3-7% for different speaker and acoustic environment scenarios over an optimized single FLSTM model, while retaining a similar computational footprint.

Related papers

• Unlocking the Power of LSTM for Long Term Time Series Forecasting [27.245021350821638]
  We propose a simple yet efficient algorithm named P-sLSTM built upon sLSTM by incorporating patching and channel independence. These modifications substantially enhance sLSTM's performance in TSF, achieving state-of-the-art results.
  arXiv  Detail & Related papers  (2024-08-19T13:59:26Z)
• BiLSTM and Attention-Based Modulation Classification of Realistic Wireless Signals [2.0650230600617534]
  The proposed model exploits multiple representations of the wireless signal as inputs to the network. An attention layer is used after the BiLSTM layer to emphasize the important temporal features. The experimental results on the recent and realistic RML22 dataset demonstrate the superior performance of the proposed model with an accuracy up to around 99%.
  arXiv  Detail & Related papers  (2024-08-14T01:17:19Z)
• RigLSTM: Recurrent Independent Grid LSTM for Generalizable Sequence Learning [75.61681328968714]
  We propose recurrent independent Grid LSTM (RigLSTM) to exploit the underlying modular structure of the target task. Our model adopts cell selection, input feature selection, hidden state selection, and soft state updating to achieve a better generalization ability.
  arXiv  Detail & Related papers  (2023-11-03T07:40:06Z)
• MAST: Multiscale Audio Spectrogram Transformers [53.06337011259031]
  We present Multiscale Audio Spectrogram Transformer (MAST) for audio classification, which brings the concept of multiscale feature hierarchies to the Audio Spectrogram Transformer (AST) In practice, MAST significantly outperforms AST by an average accuracy of 3.4% across 8 speech and non-speech tasks from the LAPE Benchmark.
  arXiv  Detail & Related papers  (2022-11-02T23:34:12Z)
• Image Classification using Sequence of Pixels [3.04585143845864]
  This study compares sequential image classification methods based on recurrent neural networks. We describe methods based on Long-Short-Term memory(LSTM), bidirectional Long-Short-Term memory(BiLSTM) architectures, etc.
  arXiv  Detail & Related papers  (2022-09-23T09:42:44Z)
• A Multi-Stage Multi-Codebook VQ-VAE Approach to High-Performance Neural TTS [52.51848317549301]
  We propose a Multi-Stage, Multi-Codebook (MSMC) approach to high-performance neural TTS synthesis. A vector-quantized, variational autoencoder (VQ-VAE) based feature analyzer is used to encode Mel spectrograms of speech training data. In synthesis, the neural vocoder converts the predicted MSMCRs into final speech waveforms.
  arXiv  Detail & Related papers  (2022-09-22T09:43:17Z)
• Bayesian Neural Network Language Modeling for Speech Recognition [59.681758762712754]
  State-of-the-art neural network language models (NNLMs) represented by long short term memory recurrent neural networks (LSTM-RNNs) and Transformers are becoming highly complex. In this paper, an overarching full Bayesian learning framework is proposed to account for the underlying uncertainty in LSTM-RNN and Transformer LMs.
  arXiv  Detail & Related papers  (2022-08-28T17:50:19Z)
• TMS: A Temporal Multi-scale Backbone Design for Speaker Embedding [60.292702363839716]
  Current SOTA backbone networks for speaker embedding are designed to aggregate multi-scale features from an utterance with multi-branch network architectures for speaker representation. We propose an effective temporal multi-scale (TMS) model where multi-scale branches could be efficiently designed in a speaker embedding network almost without increasing computational costs.
  arXiv  Detail & Related papers  (2022-03-17T05:49:35Z)
• Streaming Multi-Talker ASR with Token-Level Serialized Output Training [53.11450530896623]
  t-SOT is a novel framework for streaming multi-talker automatic speech recognition. The t-SOT model has the advantages of less inference cost and a simpler model architecture. For non-overlapping speech, the t-SOT model is on par with a single-talker ASR model in terms of both accuracy and computational cost.
  arXiv  Detail & Related papers  (2022-02-02T01:27:21Z)
• Enhancement of Spatial Clustering-Based Time-Frequency Masks using LSTM Neural Networks [3.730592618611028]
  We use LSTMs to enhance spatial clustering based time-frequency masks. We achieve both the signal modeling performance of multiple single-channel LSTM-DNN speech enhancers and the signal separation performance. We evaluate the intelligibility of the output of each system using word error rate from a Kaldi automatic speech recognizer.
  arXiv  Detail & Related papers  (2020-12-02T22:29:29Z)
• Transformer in action: a comparative study of transformer-based acoustic models for large scale speech recognition applications [23.470690511056173]
  We compare transformer based acoustic models with their LSTM counterparts on industrial scale tasks. On a low latency voice assistant task, Emformer gets 24% to 26% relative word error rate reductions (WERRs) For medium latency scenarios, comparing with LCBLSTM with similar model size and latency, Emformer gets significant WERR across four languages in video captioning datasets.
  arXiv  Detail & Related papers  (2020-10-27T23:04:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.