A Deep Learning System for Domain-specific Speech Recognition

• URL: http://arxiv.org/abs/2303.10510v2
• Date: Wed, 27 Sep 2023 17:32:30 GMT
• Title: A Deep Learning System for Domain-specific Speech Recognition
• Authors: Yanan Jia
• Abstract summary: The author works with pre-trained DeepSpeech2 and Wav2Vec2 acoustic models to develop benefit-specific ASR systems. The best performance comes from a fine-tuned Wav2Vec2-Large-LV60 acoustic model with an external KenLM. The viability of using error prone ASR transcriptions as part of spoken language understanding (SLU) is also investigated.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As human-machine voice interfaces provide easy access to increasingly intelligent machines, many state-of-the-art automatic speech recognition (ASR) systems are proposed. However, commercial ASR systems usually have poor performance on domain-specific speech especially under low-resource settings. The author works with pre-trained DeepSpeech2 and Wav2Vec2 acoustic models to develop benefit-specific ASR systems. The domain-specific data are collected using proposed semi-supervised learning annotation with little human intervention. The best performance comes from a fine-tuned Wav2Vec2-Large-LV60 acoustic model with an external KenLM, which surpasses the Google and AWS ASR systems on benefit-specific speech. The viability of using error prone ASR transcriptions as part of spoken language understanding (SLU) is also investigated. Results of a benefit-specific natural language understanding (NLU) task show that the domain-specific fine-tuned ASR system can outperform the commercial ASR systems even when its transcriptions have higher word error rate (WER), and the results between fine-tuned ASR and human transcriptions are similar.

Related papers

• Large Language Models Are Strong Audio-Visual Speech Recognition Learners [53.142635674428874]
  Multimodal large language models (MLLMs) have recently become a focal point of research due to their formidable multimodal understanding capabilities. We propose Llama-AVSR, a new MLLM with strong audio-visual speech recognition capabilities. We evaluate our proposed approach on LRS3, the largest public AVSR benchmark, and we achieve new state-of-the-art results for the tasks of ASR and AVSR with a WER of 0.81% and 0.77%, respectively.
  arXiv  Detail & Related papers  (2024-09-18T21:17:27Z)
• Towards interfacing large language models with ASR systems using confidence measures and prompting [54.39667883394458]
  This work investigates post-hoc correction of ASR transcripts with large language models (LLMs) To avoid introducing errors into likely accurate transcripts, we propose a range of confidence-based filtering methods. Our results indicate that this can improve the performance of less competitive ASR systems.
  arXiv  Detail & Related papers  (2024-07-31T08:00:41Z)
• Speculative Speech Recognition by Audio-Prefixed Low-Rank Adaptation of Language Models [21.85677682584916]
  speculative speech recognition (SSR) We propose a model which does SSR by combining a RNN-Transducer-based ASR system with an audio-ed language model (LM)
  arXiv  Detail & Related papers  (2024-07-05T16:52:55Z)
• Deliberation Model for On-Device Spoken Language Understanding [69.5587671262691]
  We propose a novel deliberation-based approach to end-to-end (E2E) spoken language understanding (SLU) We show that our approach can significantly reduce the degradation when moving from natural speech to synthetic speech training.
  arXiv  Detail & Related papers  (2022-04-04T23:48:01Z)
• ASR-Aware End-to-end Neural Diarization [15.172086811068962]
  We present a Conformer-based end-to-end neural diarization (EEND) model that uses both acoustic input and features derived from an automatic speech recognition (ASR) model. Three modifications to the Conformer-based EEND architecture are proposed to incorporate the features. Experiments on the two-speaker English conversations of Switchboard+SRE data sets show that multi-task learning with position-in-word information is the most effective way of utilizing ASR features.
  arXiv  Detail & Related papers  (2022-02-02T21:17:14Z)
• Automatic Speech Recognition using limited vocabulary: A survey [0.0]
  An approach to design an ASR system targeting under-resourced languages is to start with a limited vocabulary. This paper aims to provide a comprehensive view of mechanisms behind ASR systems as well as techniques, tools, projects, recent contributions, and possibly future directions in ASR using a limited vocabulary.
  arXiv  Detail & Related papers  (2021-08-23T15:51:41Z)
• On the Impact of Word Error Rate on Acoustic-Linguistic Speech Emotion Recognition: An Update for the Deep Learning Era [0.0]
  We create transcripts from the original speech by applying three modern ASR systems. For extraction and learning of acoustic speech features, we utilise openSMILE, openXBoW, DeepSpectrum, and auDeep. We achieve state-of-the-art unweighted average recall values of $73.6,%$ and $73.8,%$ on the speaker-independent development and test partitions of IEMOCAP.
  arXiv  Detail & Related papers  (2021-04-20T17:10:01Z)
• Dynamic Acoustic Unit Augmentation With BPE-Dropout for Low-Resource End-to-End Speech Recognition [62.94773371761236]
  We consider building an effective end-to-end ASR system in low-resource setups with a high OOV rate. We propose a method of dynamic acoustic unit augmentation based on the BPE-dropout technique. Our monolingual Turkish Conformer established a competitive result with 22.2% character error rate (CER) and 38.9% word error rate (WER)
  arXiv  Detail & Related papers  (2021-03-12T10:10:13Z)
• Long-Running Speech Recognizer:An End-to-End Multi-Task Learning Framework for Online ASR and VAD [10.168591454648123]
  This paper presents a novel end-to-end (E2E), multi-task learning (MTL) framework that integrates ASR and VAD into one model. The proposed system, which we refer to as Long-Running Speech Recognizer (LR-SR), learns ASR and VAD jointly from two seperate task-specific datasets in the training stage. In the inference stage, the LR-SR system removes non-speech parts at low computational cost and recognizes speech parts with high robustness.
  arXiv  Detail & Related papers  (2021-03-02T11:49:03Z)
• End-to-end Named Entity Recognition from English Speech [51.22888702264816]
  We introduce a first publicly available NER annotated dataset for English speech and present an E2E approach, which jointly optimize the ASR and NER tagger components. We also discuss how NER from speech can be used to handle out of vocabulary (OOV) words in an ASR system.
  arXiv  Detail & Related papers  (2020-05-22T13:39:14Z)
• Improving Readability for Automatic Speech Recognition Transcription [50.86019112545596]
  We propose a novel NLP task called ASR post-processing for readability (APR) APR aims to transform the noisy ASR output into a readable text for humans and downstream tasks while maintaining the semantic meaning of the speaker. We compare fine-tuned models based on several open-sourced and adapted pre-trained models with the traditional pipeline method.
  arXiv  Detail & Related papers  (2020-04-09T09:26:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.