Automatic speech recognition for the Nepali language using CNN, bidirectional LSTM and ResNet

• URL: http://arxiv.org/abs/2406.17825v1
• Date: Tue, 25 Jun 2024 12:14:01 GMT
• Title: Automatic speech recognition for the Nepali language using CNN, bidirectional LSTM and ResNet
• Authors: Manish Dhakal, Arman Chhetri, Aman Kumar Gupta, Prabin Lamichhane, Suraj Pandey, Subarna Shakya,
• Abstract summary: This paper presents an end-to-end deep learning model for Automatic Speech Recognition (ASR) that transcribes Nepali speech to text. The model was trained and tested on the OpenSLR (audio, text) dataset. The character error rate (CER) of 17.06 percent has been achieved.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper presents an end-to-end deep learning model for Automatic Speech Recognition (ASR) that transcribes Nepali speech to text. The model was trained and tested on the OpenSLR (audio, text) dataset. The majority of the audio dataset have silent gaps at both ends which are clipped during dataset preprocessing for a more uniform mapping of audio frames and their corresponding texts. Mel Frequency Cepstral Coefficients (MFCCs) are used as audio features to feed into the model. The model having Bidirectional LSTM paired with ResNet and one-dimensional CNN produces the best results for this dataset out of all the models (neural networks with variations of LSTM, GRU, CNN, and ResNet) that have been trained so far. This novel model uses Connectionist Temporal Classification (CTC) function for loss calculation during training and CTC beam search decoding for predicting characters as the most likely sequence of Nepali text. On the test dataset, the character error rate (CER) of 17.06 percent has been achieved. The source code is available at: https://github.com/manishdhakal/ASR-Nepali-using-CNN-BiLSTM-ResNet.

Related papers

• Cross-Speaker Encoding Network for Multi-Talker Speech Recognition [74.97576062152709]
  Cross-MixSpeaker. Network addresses limitations of SIMO models by aggregating cross-speaker representations. Network is integrated with SOT to leverage both the advantages of SIMO and SISO.
  arXiv  Detail & Related papers  (2024-01-08T16:37:45Z)
• Nepali Video Captioning using CNN-RNN Architecture [0.0]
  This article presents a study on Nepali video captioning using deep neural networks. Through the integration of pre-trained CNNs and RNNs, the research focuses on generating precise and contextually relevant captions for Nepali videos. The approach involves dataset collection, data preprocessing, model implementation, and evaluation.
  arXiv  Detail & Related papers  (2023-11-05T16:09:40Z)
• ATGNN: Audio Tagging Graph Neural Network [25.78859233831268]
  ATGNN is a graph neural architecture that maps semantic relationships between learnable class embeddings and spectrogram regions. We evaluate ATGNN on two audio tagging tasks, where it achieves 0.585 mAP on the FSD50K dataset and 0.335 mAP on the AudioSet-balanced dataset.
  arXiv  Detail & Related papers  (2023-11-02T18:19:26Z)
• Mega-TTS: Zero-Shot Text-to-Speech at Scale with Intrinsic Inductive Bias [71.94109664001952]
  Mega-TTS is a novel zero-shot TTS system that is trained with large-scale wild data. We show that Mega-TTS surpasses state-of-the-art TTS systems on zero-shot TTS speech editing, and cross-lingual TTS tasks.
  arXiv  Detail & Related papers  (2023-06-06T08:54:49Z)
• Efficiently Trained Low-Resource Mongolian Text-to-Speech System Based On FullConv-TTS [0.0]
  We propose a new text-to-speech system based on deep convolutional neural networks that does not employ any RNN components (recurrent units) At the same time, we improve the generality and robustness of our model through a series of data augmentation methods such as Time Warping, Frequency Mask, and Time Mask. The final experimental results show that the TTS model using only the CNN component can reduce the training time compared to the classic TTS models such as Tacotron.
  arXiv  Detail & Related papers  (2022-10-24T14:18:43Z)
• Bayesian Neural Network Language Modeling for Speech Recognition [59.681758762712754]
  State-of-the-art neural network language models (NNLMs) represented by long short term memory recurrent neural networks (LSTM-RNNs) and Transformers are becoming highly complex. In this paper, an overarching full Bayesian learning framework is proposed to account for the underlying uncertainty in LSTM-RNN and Transformer LMs.
  arXiv  Detail & Related papers  (2022-08-28T17:50:19Z)
• R-MelNet: Reduced Mel-Spectral Modeling for Neural TTS [1.8927791081850118]
  This paper introduces R-MelNet, a two-part autoregressive architecture with a backend WaveRNN-style audio decoder. The model produces low-resolution mel-spectral features which are used by a WaveRNN decoder to produce an audio waveform.
  arXiv  Detail & Related papers  (2022-06-30T13:29:31Z)
• Rescoring Sequence-to-Sequence Models for Text Line Recognition with CTC-Prefixes [0.0]
  We propose to use the CTC-Prefix-Score during S2S decoding. During beam search, paths that are invalid according to the CTC confidence matrix are penalised. We evaluate this setup on three HTR data sets: IAM, Rimes, and StAZH.
  arXiv  Detail & Related papers  (2021-10-12T11:40:05Z)
• On Addressing Practical Challenges for RNN-Transduce [72.72132048437751]
  We adapt a well-trained RNN-T model to a new domain without collecting the audio data. We obtain word-level confidence scores by utilizing several types of features calculated during decoding. The proposed time stamping method can get less than 50ms word timing difference on average.
  arXiv  Detail & Related papers  (2021-04-27T23:31:43Z)
• Decoupling Pronunciation and Language for End-to-end Code-switching Automatic Speech Recognition [66.47000813920617]
  We propose a decoupled transformer model to use monolingual paired data and unpaired text data. The model is decoupled into two parts: audio-to-phoneme (A2P) network and phoneme-to-text (P2T) network. By using monolingual data and unpaired text data, the decoupled transformer model reduces the high dependency on code-switching paired training data of E2E model.
  arXiv  Detail & Related papers  (2020-10-28T07:46:15Z)
• Unsupervised Cross-Modal Audio Representation Learning from Unstructured Multilingual Text [69.55642178336953]
  We present an approach to unsupervised audio representation learning. Based on a triplet neural network architecture, we harnesses semantically related cross-modal information to estimate audio track-relatedness. We show that our approach is invariant to the variety of annotation styles as well as to the different languages of this collection.
  arXiv  Detail & Related papers  (2020-03-27T07:37:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.