Deep Learning for Prominence Detection in Children's Read Speech

• URL: http://arxiv.org/abs/2104.05488v2
• Date: Tue, 13 Apr 2021 04:31:35 GMT
• Title: Deep Learning for Prominence Detection in Children's Read Speech
• Authors: Kamini Sabu, Mithilesh Vaidya, Preeti Rao
• Abstract summary: We consider a labeled dataset of children's reading recordings for the speaker-independent detection of prominent words. A previous well-tuned random forest ensemble predictor is replaced by an RNN sequence to exploit potential context dependency. Deep learning is applied to obtain word-level features from low-level acoustic contours of fundamental frequency, intensity and spectral shape.
• Score: 13.041607703862724
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Expressive reading, considered the defining attribute of oral reading fluency, comprises the prosodic realization of phrasing and prominence. In the context of evaluating oral reading, it helps to establish the speaker's comprehension of the text. We consider a labeled dataset of children's reading recordings for the speaker-independent detection of prominent words using acoustic-prosodic and lexico-syntactic features. A previous well-tuned random forest ensemble predictor is replaced by an RNN sequence classifier to exploit potential context dependency across the longer utterance. Further, deep learning is applied to obtain word-level features from low-level acoustic contours of fundamental frequency, intensity and spectral shape in an end-to-end fashion. Performance comparisons are presented across the different feature types and across different feature learning architectures for prominent word prediction to draw insights wherever possible.

Related papers

• Speech Rhythm-Based Speaker Embeddings Extraction from Phonemes and Phoneme Duration for Multi-Speaker Speech Synthesis [16.497022070614236]
  This paper proposes a speech rhythm-based method for speaker embeddings to model phoneme duration using a few utterances by the target speaker. A novel feature of the proposed method is the rhythm-based embeddings extracted from phonemes and their durations, which are known to be related to speaking rhythm.
  arXiv  Detail & Related papers  (2024-02-11T02:26:43Z)
• Paralinguistics-Enhanced Large Language Modeling of Spoken Dialogue [71.15186328127409]
  Paralinguistics-enhanced Generative Pretrained Transformer (ParalinGPT) Model takes the conversational context of text, speech embeddings, and paralinguistic attributes as input prompts within a serialized multitasking framework. We utilize the Switchboard-1 corpus, including its sentiment labels as the paralinguistic attribute, as our spoken dialogue dataset.
  arXiv  Detail & Related papers  (2023-12-23T18:14:56Z)
• Learning Disentangled Speech Representations [0.412484724941528]
  SynSpeech is a novel large-scale synthetic speech dataset designed to enable research on disentangled speech representations. We present a framework to evaluate disentangled representation learning techniques, applying both linear probing and established supervised disentanglement metrics. We find that SynSpeech facilitates benchmarking across a range of factors, achieving promising disentanglement of simpler features like gender and speaking style, while highlighting challenges in isolating complex attributes like speaker identity.
  arXiv  Detail & Related papers  (2023-11-04T04:54:17Z)
• Audio-Visual Neural Syntax Acquisition [91.14892278795892]
  We study phrase structure induction from visually-grounded speech. We present the Audio-Visual Neural Syntax Learner (AV-NSL) that learns phrase structure by listening to audio and looking at images, without ever being exposed to text.
  arXiv  Detail & Related papers  (2023-10-11T16:54:57Z)
• Learning Separable Hidden Unit Contributions for Speaker-Adaptive Lip-Reading [73.59525356467574]
  A speaker's own characteristics can always be portrayed well by his/her few facial images or even a single image with shallow networks. Fine-grained dynamic features associated with speech content expressed by the talking face always need deep sequential networks. Our approach consistently outperforms existing methods.
  arXiv  Detail & Related papers  (2023-10-08T07:48:25Z)
• Learning utterance-level representations through token-level acoustic latents prediction for Expressive Speech Synthesis [3.691712391306624]
  We show that the fine-grained latent space also captures coarse-grained information, which is more evident as the dimension of latent space increases in order to capture diverse prosodic representations. We alleviate this issue by first capturing rich speech attributes into a token-level latent space and then, separately train a prior network that given the input text, learns utterance-level representations in order to predict the phoneme-level, posterior latents extracted during the previous step.
  arXiv  Detail & Related papers  (2022-11-01T15:17:25Z)
• Deep Neural Convolutive Matrix Factorization for Articulatory Representation Decomposition [48.56414496900755]
  This work uses a neural implementation of convolutive sparse matrix factorization to decompose the articulatory data into interpretable gestures and gestural scores. Phoneme recognition experiments were additionally performed to show that gestural scores indeed code phonological information successfully.
  arXiv  Detail & Related papers  (2022-04-01T14:25:19Z)
• Deep Learning For Prominence Detection In Children's Read Speech [13.041607703862724]
  We present a system that operates on segmented speech waveforms to learn features relevant to prominent word detection for children's oral fluency assessment. The chosen CRNN (convolutional recurrent neural network) framework, incorporating both word-level features and sequence information, is found to benefit from the perceptually motivated SincNet filters.
  arXiv  Detail & Related papers  (2021-10-27T08:51:42Z)
• Leveraging Acoustic and Linguistic Embeddings from Pretrained speech and language Models for Intent Classification [81.80311855996584]
  We propose a novel intent classification framework that employs acoustic features extracted from a pretrained speech recognition system and linguistic features learned from a pretrained language model. We achieve 90.86% and 99.07% accuracy on ATIS and Fluent speech corpus, respectively.
  arXiv  Detail & Related papers  (2021-02-15T07:20:06Z)
• Speaker Diarization with Lexical Information [59.983797884955]
  This work presents a novel approach for speaker diarization to leverage lexical information provided by automatic speech recognition. We propose a speaker diarization system that can incorporate word-level speaker turn probabilities with speaker embeddings into a speaker clustering process to improve the overall diarization accuracy.
  arXiv  Detail & Related papers  (2020-04-13T17:16:56Z)
• Identification of primary and collateral tracks in stuttered speech [22.921077940732]
  We introduce a new evaluation framework for disfluency detection inspired by the clinical and NLP perspective. We present a novel forced-aligned disfluency dataset from a corpus of semi-directed interviews. We show experimentally that using word-based span features outperformed the baselines for speech-based predictions.
  arXiv  Detail & Related papers  (2020-03-02T16:50:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.