Quartered Spectral Envelope and 1D-CNN-based Classification of Normally Phonated and Whispered Speech

• URL: http://arxiv.org/abs/2408.13746v1
• Date: Sun, 25 Aug 2024 07:17:11 GMT
• Title: Quartered Spectral Envelope and 1D-CNN-based Classification of Normally Phonated and Whispered Speech
• Authors: S. Johanan Joysingh, P. Vijayalakshmi, T. Nagarajan,
• Abstract summary: The presence of pitch and pitch harmonics in normal speech, and its absence in whispered speech, is evident in the spectral envelope of the Fourier transform. We propose the use of one dimensional convolutional neural networks (1D-CNN) to capture these features. The system yields an accuracy of 99.31% when trained and tested on the wTIMIT dataset, and 100% on the CHAINS dataset.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Whisper, as a form of speech, is not sufficiently addressed by mainstream speech applications. This is due to the fact that systems built for normal speech do not work as expected for whispered speech. A first step to building a speech application that is inclusive of whispered speech, is the successful classification of whispered speech and normal speech. Such a front-end classification system is expected to have high accuracy and low computational overhead, which is the scope of this paper. One of the characteristics of whispered speech is the absence of the fundamental frequency (or pitch), and hence the pitch harmonics as well. The presence of the pitch and pitch harmonics in normal speech, and its absence in whispered speech, is evident in the spectral envelope of the Fourier transform. We observe that this characteristic is predominant in the first quarter of the spectrum, and exploit the same as a feature. We propose the use of one dimensional convolutional neural networks (1D-CNN) to capture these features from the quartered spectral envelope (QSE). The system yields an accuracy of 99.31% when trained and tested on the wTIMIT dataset, and 100% on the CHAINS dataset. The proposed feature is compared with Mel frequency cepstral coefficients (MFCC), a staple in the speech domain. The proposed classification system is also compared with the state-of-the-art system based on log-filterbank energy (LFBE) features trained on long short-term memory (LSTM) network. The proposed system based on 1D-CNN performs better than, or as good as, the state-of-the-art across multiple experiments. It also converges sooner, with lesser computational overhead. Finally, the proposed system is evaluated under the presence of white noise at various signal-to-noise ratios and found to be robust.

Related papers

• Quartered Chirp Spectral Envelope for Whispered vs Normal Speech Classification [0.0]
  We propose a new feature named the quartered chirp spectral envelope to classify whispered and normal speech. The feature is trained on a one dimensional convolutional neural network, that captures the trends in the spectral envelope. The proposed system performs better than the state of the art, in the presence of white noise.
  arXiv  Detail & Related papers  (2024-08-27T04:56:22Z)
• Clustering and Mining Accented Speech for Inclusive and Fair Speech Recognition [18.90193320368228]
  We present accent clustering and mining schemes for fair speech recognition systems. For accent recognition, we applied three schemes to overcome limited size of supervised accent data. Fine-tuning ASR on the mined Indian accent speech showed 10.0% and 5.3% relative improvements compared to fine-tuning on the randomly sampled speech.
  arXiv  Detail & Related papers  (2024-08-05T16:00:07Z)
• Syllable based DNN-HMM Cantonese Speech to Text System [3.976127530758402]
  This paper builds up a Cantonese Speech-to-Text (STT) system with a syllable based acoustic model. The ONC-based syllable acoustic modeling achieves the best performance with the word error rate (WER) of 9.66% and the real time factor (RTF) of 1.38812.
  arXiv  Detail & Related papers  (2024-02-13T20:54:24Z)
• Audio-visual End-to-end Multi-channel Speech Separation, Dereverberation and Recognition [52.11964238935099]
  An audio-visual multi-channel speech separation, dereverberation and recognition approach is proposed in this paper. Video input is consistently demonstrated in mask-based MVDR speech separation, DNN-WPE or spectral mapping (SpecM) based speech dereverberation front-end. Experiments were conducted on the mixture overlapped and reverberant speech data constructed using simulation or replay of the Oxford LRS2 dataset.
  arXiv  Detail & Related papers  (2023-07-06T10:50:46Z)
• TranSpeech: Speech-to-Speech Translation With Bilateral Perturbation [61.564874831498145]
  TranSpeech is a speech-to-speech translation model with bilateral perturbation. We establish a non-autoregressive S2ST technique, which repeatedly masks and predicts unit choices. TranSpeech shows a significant improvement in inference latency, enabling speedup up to 21.4x than autoregressive technique.
  arXiv  Detail & Related papers  (2022-05-25T06:34:14Z)
• Speech-enhanced and Noise-aware Networks for Robust Speech Recognition [25.279902171523233]
  A noise-aware training framework based on two cascaded neural structures is proposed to jointly optimize speech enhancement and speech recognition. The two proposed systems achieve word error rate (WER) of 3.90% and 3.55%, respectively, on the Aurora-4 task. Compared with the best existing systems that use bigram and trigram language models for decoding, the proposed CNN-TDNNF-based system achieves a relative WER reduction of 15.20% and 33.53%, respectively.
  arXiv  Detail & Related papers  (2022-03-25T15:04:51Z)
• Speaker Embedding-aware Neural Diarization: a Novel Framework for Overlapped Speech Diarization in the Meeting Scenario [51.5031673695118]
  We reformulate overlapped speech diarization as a single-label prediction problem. We propose the speaker embedding-aware neural diarization (SEND) system.
  arXiv  Detail & Related papers  (2022-03-18T06:40:39Z)
• Real-time Speaker counting in a cocktail party scenario using Attention-guided Convolutional Neural Network [60.99112031408449]
  We propose a real-time, single-channel attention-guided Convolutional Neural Network (CNN) to estimate the number of active speakers in overlapping speech. The proposed system extracts higher-level information from the speech spectral content using a CNN model. Experiments on simulated overlapping speech using WSJ corpus show that the attention solution is shown to improve the performance by almost 3% absolute over conventional temporal average pooling.
  arXiv  Detail & Related papers  (2021-10-30T19:24:57Z)
• Training Speech Enhancement Systems with Noisy Speech Datasets [7.157870452667369]
  We propose two improvements to train SE systems on noisy speech data. First, we propose several modifications of the loss functions, which make them robust against noisy speech targets. We show that using our robust loss function improves PESQ by up to 0.19 compared to a system trained in the traditional way.
  arXiv  Detail & Related papers  (2021-05-26T03:32:39Z)
• Any-to-Many Voice Conversion with Location-Relative Sequence-to-Sequence Modeling [61.351967629600594]
  This paper proposes an any-to-many location-relative, sequence-to-sequence (seq2seq), non-parallel voice conversion approach. In this approach, we combine a bottle-neck feature extractor (BNE) with a seq2seq synthesis module. Objective and subjective evaluations show that the proposed any-to-many approach has superior voice conversion performance in terms of both naturalness and speaker similarity.
  arXiv  Detail & Related papers  (2020-09-06T13:01:06Z)
• End-to-end Whispered Speech Recognition with Frequency-weighted Approaches and Pseudo Whisper Pre-training [130.56878980058966]
  We present several approaches for end-to-end (E2E) recognition of whispered speech. We achieve an overall relative reduction of 19.8% in PER and 44.4% in CER on a relatively small whispered TIMIT corpus. As long as we have a good E2E model pre-trained on normal or pseudo-whispered speech, a relatively small set of whispered speech may suffice to obtain a reasonably good E2E whispered speech recognizer.
  arXiv  Detail & Related papers  (2020-05-05T07:08:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.