High Performance Sequence-to-Sequence Model for Streaming Speech Recognition

• URL: http://arxiv.org/abs/2003.10022v2
• Date: Sun, 26 Jul 2020 21:32:22 GMT
• Title: High Performance Sequence-to-Sequence Model for Streaming Speech Recognition
• Authors: Thai-Son Nguyen, Ngoc-Quan Pham, Sebastian Stueker, Alex Waibel
• Abstract summary: Sequence-to-sequence models have started to achieve state-of-the-art performance on standard speech recognition tasks. But when it comes to performing run-on recognition on an input stream of audio data, these models face several challenges. We introduce an additional loss function controlling the uncertainty of the attention mechanism, a modified beam search identifying partial, stable hypotheses, ways of working with BLSTM in the encoder, and the use of chunked BLSTM.
• Score: 19.488757267198498
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently sequence-to-sequence models have started to achieve state-of-the-art performance on standard speech recognition tasks when processing audio data in batch mode, i.e., the complete audio data is available when starting processing. However, when it comes to performing run-on recognition on an input stream of audio data while producing recognition results in real-time and with low word-based latency, these models face several challenges. For many techniques, the whole audio sequence to be decoded needs to be available at the start of the processing, e.g., for the attention mechanism or the bidirectional LSTM (BLSTM). In this paper, we propose several techniques to mitigate these problems. We introduce an additional loss function controlling the uncertainty of the attention mechanism, a modified beam search identifying partial, stable hypotheses, ways of working with BLSTM in the encoder, and the use of chunked BLSTM. Our experiments show that with the right combination of these techniques, it is possible to perform run-on speech recognition with low word-based latency without sacrificing in word error rate performance.

Related papers

• SONAR: A Synthetic AI-Audio Detection Framework and Benchmark [59.09338266364506]
  SONAR is a synthetic AI-Audio Detection Framework and Benchmark. It aims to provide a comprehensive evaluation for distinguishing cutting-edge AI-synthesized auditory content. It is the first framework to uniformly benchmark AI-audio detection across both traditional and foundation model-based deepfake detection systems.
  arXiv  Detail & Related papers  (2024-10-06T01:03:42Z)
• Qifusion-Net: Layer-adapted Stream/Non-stream Model for End-to-End Multi-Accent Speech Recognition [1.0690007351232649]
  We propose a layer-adapted fusion (LAF) model, called Qifusion-Net, which does not require any prior knowledge about the target accent. Experiment results demonstrate that our proposed methods outperform the baseline with relative reductions of 22.1$%$ and 17.2$%$ in character error rate (CER) across multi accent test datasets.
  arXiv  Detail & Related papers  (2024-07-03T11:35:52Z)
• Autoregressive Diffusion Transformer for Text-to-Speech Synthesis [39.32761051774537]
  We propose encoding audio as vector sequences in continuous space $mathbb Rd$ and autoregressively generating these sequences. High-bitrate continuous speech representation enables almost flawless reconstruction, allowing our model to achieve nearly perfect speech editing.
  arXiv  Detail & Related papers  (2024-06-08T18:57:13Z)
• DiffSED: Sound Event Detection with Denoising Diffusion [70.18051526555512]
  We reformulate the SED problem by taking a generative learning perspective. Specifically, we aim to generate sound temporal boundaries from noisy proposals in a denoising diffusion process. During training, our model learns to reverse the noising process by converting noisy latent queries to the groundtruth versions.
  arXiv  Detail & Related papers  (2023-08-14T17:29:41Z)
• Simple Pooling Front-ends For Efficient Audio Classification [56.59107110017436]
  We show that eliminating the temporal redundancy in the input audio features could be an effective approach for efficient audio classification. We propose a family of simple pooling front-ends (SimPFs) which use simple non-parametric pooling operations to reduce the redundant information. SimPFs can achieve a reduction in more than half of the number of floating point operations for off-the-shelf audio neural networks.
  arXiv  Detail & Related papers  (2022-10-03T14:00:41Z)
• STOP: A dataset for Spoken Task Oriented Semantic Parsing [66.14615249745448]
  End-to-end spoken language understanding (SLU) predicts intent directly from audio using a single model. We release the Spoken Task-Oriented semantic Parsing (STOP) dataset, the largest and most complex SLU dataset to be publicly available. In addition to the human-recorded audio, we are releasing a TTS-generated version to benchmark the performance for low-resource domain adaptation of end-to-end SLU systems.
  arXiv  Detail & Related papers  (2022-06-29T00:36:34Z)
• Advances in Speech Vocoding for Text-to-Speech with Continuous Parameters [2.6572330982240935]
  This paper presents new techniques in a continuous vocoder, that is all features are continuous and presents a flexible speech synthesis system. New continuous noise masking based on the phase distortion is proposed to eliminate the perceptual impact of the residual noise. Bidirectional long short-term memory (LSTM) and gated recurrent unit (GRU) are studied and applied to model continuous parameters for more natural-sounding like a human.
  arXiv  Detail & Related papers  (2021-06-19T12:05:01Z)
• Dissecting User-Perceived Latency of On-Device E2E Speech Recognition [34.645194215436966]
  We show that factors affecting token emission latency, and endpointing behavior significantly impact on user-perceived latency (UPL) We achieve the best trade-off between latency and word error rate when performing ASR jointly with endpointing, and using the recently proposed alignment regularization.
  arXiv  Detail & Related papers  (2021-04-06T00:55:11Z)
• Streaming end-to-end multi-talker speech recognition [34.76106500736099]
  We propose the Streaming Unmixing and Recognition Transducer (SURT) for end-to-end multi-talker speech recognition. Our model employs the Recurrent Neural Network Transducer (RNN-T) as the backbone that can meet various latency constraints. Based on experiments on the publicly available LibriSpeechMix dataset, we show that HEAT can achieve better accuracy compared with PIT.
  arXiv  Detail & Related papers  (2020-11-26T06:28:04Z)
• VoiceFilter-Lite: Streaming Targeted Voice Separation for On-Device Speech Recognition [60.462770498366524]
  We introduce VoiceFilter-Lite, a single-channel source separation model that runs on the device to preserve only the speech signals from a target user. We show that such a model can be quantized as a 8-bit integer model and run in realtime.
  arXiv  Detail & Related papers  (2020-09-09T14:26:56Z)
• Any-to-Many Voice Conversion with Location-Relative Sequence-to-Sequence Modeling [61.351967629600594]
  This paper proposes an any-to-many location-relative, sequence-to-sequence (seq2seq), non-parallel voice conversion approach. In this approach, we combine a bottle-neck feature extractor (BNE) with a seq2seq synthesis module. Objective and subjective evaluations show that the proposed any-to-many approach has superior voice conversion performance in terms of both naturalness and speaker similarity.
  arXiv  Detail & Related papers  (2020-09-06T13:01:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.