Investigating self-supervised, weakly supervised and fully supervised training approaches for multi-domain automatic speech recognition: a study on Bangladeshi Bangla

• URL: http://arxiv.org/abs/2210.12921v3
• Date: Thu, 11 May 2023 01:06:17 GMT
• Title: Investigating self-supervised, weakly supervised and fully supervised training approaches for multi-domain automatic speech recognition: a study on Bangladeshi Bangla
• Authors: Ahnaf Mozib Samin, M. Humayon Kobir, Md. Mushtaq Shahriyar Rafee, M. Firoz Ahmed, Mehedi Hasan, Partha Ghosh, Shafkat Kibria, and M. Shahidur Rahman
• Abstract summary: Speech recognition systems still suffer from a lack of robustness and generalizability issues due to domain shifting. In this study, we investigate the robustness of the state-of-the-art transfer learning approaches such as self-supervised wav2vec 2.0 and weakly supervised Whisper. We also demonstrate the significance of domain selection while building a corpus by assessing these models on a novel multi-domain Bangladeshi Bangla ASR benchmark.
• Score: 4.869409466908974
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Despite huge improvements in automatic speech recognition (ASR) employing neural networks, ASR systems still suffer from a lack of robustness and generalizability issues due to domain shifting. This is mainly because principal corpus design criteria are often not identified and examined adequately while compiling ASR datasets. In this study, we investigate the robustness of the state-of-the-art transfer learning approaches such as self-supervised wav2vec 2.0 and weakly supervised Whisper as well as fully supervised convolutional neural networks (CNNs) for multi-domain ASR. We also demonstrate the significance of domain selection while building a corpus by assessing these models on a novel multi-domain Bangladeshi Bangla ASR evaluation benchmark - BanSpeech, which contains approximately 6.52 hours of human-annotated speech and 8085 utterances from 13 distinct domains. SUBAK.KO, a mostly read speech corpus for the morphologically rich language Bangla, has been used to train the ASR systems. Experimental evaluation reveals that self-supervised cross-lingual pre-training is the best strategy compared to weak supervision and full supervision to tackle the multi-domain ASR task. Moreover, the ASR models trained on SUBAK.KO face difficulty recognizing speech from domains with mostly spontaneous speech. The BanSpeech will be publicly available to meet the need for a challenging evaluation benchmark for Bangla ASR.

Related papers

• WESR: Scaling and Evaluating Word-level Event-Speech Recognition [59.21814194620928]
  Speech conveys not only linguistic information but also rich non-verbal vocal events such as laughing and crying.<n>We develop a refined taxonomy of 21 vocal events, with a new categorization into discrete (standalone) versus continuous (mixed with speech) types.<n>Based on the refined taxonomy, we introduce WESR-Bench, an expert-annotated evaluation set (900+ utterances) with a novel position-aware protocol.
  arXiv  Detail & Related papers  (2026-01-08T02:23:21Z)
• Recent Trends in Distant Conversational Speech Recognition: A Review of CHiME-7 and 8 DASR Challenges [58.80034860169605]
  The CHiME-7 and 8 distant speech recognition (DASR) challenges focus on multi-channel, generalizable, joint automatic speech recognition (ASR) and diarization of conversational speech.<n>This paper outlines the challenges' design, evaluation metrics, datasets, and baseline systems while analyzing key trends from participant submissions.
  arXiv  Detail & Related papers  (2025-07-24T07:56:24Z)
• Towards Unsupervised Speech Recognition Without Pronunciation Models [57.222729245842054]
  Most languages lack sufficient paired speech and text data to effectively train automatic speech recognition systems. We propose the removal of reliance on a phoneme lexicon to develop unsupervised ASR systems. We experimentally demonstrate that an unsupervised speech recognizer can emerge from joint speech-to-speech and text-to-text masked token-infilling.
  arXiv  Detail & Related papers  (2024-06-12T16:30:58Z)
• Locality enhanced dynamic biasing and sampling strategies for contextual ASR [7.640373723875947]
  Contextual biasing (CB) modules bias ASR model towards such contextually-relevant phrases. In this work we firstly analyse different sampling strategies to provide insights into the training of CB for ASR. Secondly, we introduce a neighbourhood attention (NA) that localizes self attention (SA) to the nearest neighbouring frames.
  arXiv  Detail & Related papers  (2024-01-23T23:46:01Z)
• Exploring the Integration of Speech Separation and Recognition with Self-Supervised Learning Representation [83.36685075570232]
  This work provides an insightful investigation of speech separation in reverberant and noisy-reverberant scenarios as an ASR front-end. We explore multi-channel separation methods, mask-based beamforming and complex spectral mapping, as well as the best features to use in the ASR back-end model. A proposed integration using TF-GridNet-based complex spectral mapping and WavLM-based SSLR achieves a 2.5% word error rate in reverberant WHAMR! test set.
  arXiv  Detail & Related papers  (2023-07-23T05:39:39Z)
• A Comparative Study of Speaker Role Identification in Air Traffic Communication Using Deep Learning Approaches [9.565067058593316]
  We formulate the speaker role identification (SRI) task of controller-pilot communication as a binary classification problem. To ablate the impacts of the comparative approaches, various advanced neural network architectures are applied. The proposed MMSRINet shows the competitive performance and robustness than the other methods on both seen and unseen data.
  arXiv  Detail & Related papers  (2021-11-03T07:00:20Z)
• WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing [102.45426364965887]
  We propose a new pre-trained model, WavLM, to solve full-stack downstream speech tasks. WavLM is built based on the HuBERT framework, with an emphasis on both spoken content modeling and speaker identity preservation. We scale up the training dataset from 60k hours to 94k hours of public audio data, and optimize its training procedure for better representation extraction.
  arXiv  Detail & Related papers  (2021-10-26T17:55:19Z)
• An Exploration of Self-Supervised Pretrained Representations for End-to-End Speech Recognition [98.70304981174748]
  We focus on the general applications of pretrained speech representations, on advanced end-to-end automatic speech recognition (E2E-ASR) models. We select several pretrained speech representations and present the experimental results on various open-source and publicly available corpora for E2E-ASR.
  arXiv  Detail & Related papers  (2021-10-09T15:06:09Z)
• ATCSpeechNet: A multilingual end-to-end speech recognition framework for air traffic control systems [15.527854608553824]
  ATCSpeechNet is proposed to tackle the issue of translating communication speech into human-readable text in air traffic control systems. An end-to-end paradigm is developed to convert speech waveform into text directly, without any feature engineering or lexicon. Experimental results on the ATCSpeech corpus demonstrate that the proposed approach achieves a high performance with a very small labeled corpus.
  arXiv  Detail & Related papers  (2021-02-17T02:27:09Z)
• A bandit approach to curriculum generation for automatic speech recognition [7.008190762572486]
  We present an approach to mitigate the lack of training data by employing Automated Curriculum Learning. The goal of the approach is to optimize the training sequence of mini-batches ranked by the level of difficulty. We test our approach on a truly low-resource language and show that the bandit framework has a good improvement over the baseline transfer-learning model.
  arXiv  Detail & Related papers  (2021-02-06T20:32:10Z)
• Characterizing Speech Adversarial Examples Using Self-Attention U-Net Enhancement [102.48582597586233]
  We present a U-Net based attention model, U-Net$_At$, to enhance adversarial speech signals. We conduct experiments on the automatic speech recognition (ASR) task with adversarial audio attacks.
  arXiv  Detail & Related papers  (2020-03-31T02:16:34Z)
• Improving noise robust automatic speech recognition with single-channel time-domain enhancement network [100.1041336974175]
  We show that a single-channel time-domain denoising approach can significantly improve ASR performance. We show that single-channel noise reduction can still improve ASR performance.
  arXiv  Detail & Related papers  (2020-03-09T09:36:31Z)
• Joint Contextual Modeling for ASR Correction and Language Understanding [60.230013453699975]
  We propose multi-task neural approaches to perform contextual language correction on ASR outputs jointly with language understanding (LU) We show that the error rates of off the shelf ASR and following LU systems can be reduced significantly by 14% relative with joint models trained using small amounts of in-domain data.
  arXiv  Detail & Related papers  (2020-01-28T22:09:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.