Segmentation-free Goodness of Pronunciation

• URL: http://arxiv.org/abs/2507.16838v2
• Date: Thu, 24 Jul 2025 02:55:40 GMT
• Title: Segmentation-free Goodness of Pronunciation
• Authors: Xinwei Cao, Zijian Fan, Torbjørn Svendsen, Giampiero Salvi,
• Abstract summary: Mispronunciation detection and diagnosis (MDD) is a significant part in modern computer aided language learning (CALL) systems.<n>Most systems are based on a form of goodness of pronunciation (GOP) which requires pre-segmentation of speech into phonetic units.<n>In this study, we first propose self-alignment GOP (GOP-SA) that enables the use of CTC-trained ASR models for MDD.<n>Next, we define a more general alignment-free method that takes all possible alignments of the target phoneme into account (GOP-AF)
• Score: 8.859671910259244
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Mispronunciation detection and diagnosis (MDD) is a significant part in modern computer aided language learning (CALL) systems. Within MDD, phoneme-level pronunciation assessment is key to helping L2 learners improve their pronunciation. However, most systems are based on a form of goodness of pronunciation (GOP) which requires pre-segmentation of speech into phonetic units. This limits the accuracy of these methods and the possibility to use modern CTC-based acoustic models for their evaluation. In this study, we first propose self-alignment GOP (GOP-SA) that enables the use of CTC-trained ASR models for MDD. Next, we define a more general alignment-free method that takes all possible alignments of the target phoneme into account (GOP-AF). We give a theoretical account of our definition of GOP-AF, an implementation that solves potential numerical issues as well as a proper normalization which makes the method applicable with acoustic models with different peakiness over time. We provide extensive experimental results on the CMU Kids and Speechocean762 datasets comparing the different definitions of our methods, estimating the dependency of GOP-AF on the peakiness of the acoustic models and on the amount of context around the target phoneme. Finally, we compare our methods with recent studies over the Speechocean762 data showing that the feature vectors derived from the proposed method achieve state-of-the-art results on phoneme-level pronunciation assessment.

Related papers

• $C^2$AV-TSE: Context and Confidence-aware Audio Visual Target Speaker Extraction [80.57232374640911]
  We propose a model-agnostic strategy called the Mask-And-Recover (MAR)<n>MAR integrates both inter- and intra-modality contextual correlations to enable global inference within extraction modules.<n>To better target challenging parts within each sample, we introduce a Fine-grained Confidence Score (FCS) model.
  arXiv  Detail & Related papers  (2025-04-01T13:01:30Z)
• Towards Efficient and Multifaceted Computer-assisted Pronunciation Training Leveraging Hierarchical Selective State Space Model and Decoupled Cross-entropy Loss [5.101375571703936]
  HMamba is a novel CAPT approach that seamlessly integrates APA and MDD tasks in parallel.<n>A comprehensive set of empirical results on the speechocean762 benchmark dataset demonstrates the effectiveness of our approach on APA.<n>Our proposed approach also yields a considerable improvement in MDD performance over a strong baseline, achieving an F1-score of 63.85%.
  arXiv  Detail & Related papers  (2025-02-11T14:17:29Z)
• CLAIR-A: Leveraging Large Language Models to Judge Audio Captions [73.51087998971418]
  evaluating machine-generated audio captions is a complex task that requires considering diverse factors. We propose CLAIR-A, a simple and flexible method that leverages the zero-shot capabilities of large language models. In our evaluations, CLAIR-A better predicts human judgements of quality compared to traditional metrics.
  arXiv  Detail & Related papers  (2024-09-19T17:59:52Z)
• Phonological Level wav2vec2-based Mispronunciation Detection and Diagnosis Method [11.069975459609829]
  We propose a low-level Mispronunciation Detection and Diagnosis (MDD) approach based on the detection of speech attribute features. The proposed method was applied to L2 speech corpora collected from English learners from different native languages.
  arXiv  Detail & Related papers  (2023-11-13T02:41:41Z)
• Learning Phone Recognition from Unpaired Audio and Phone Sequences Based on Generative Adversarial Network [58.82343017711883]
  This paper investigates how to learn directly from unpaired phone sequences and speech utterances. GAN training is adopted in the first stage to find the mapping relationship between unpaired speech and phone sequence. In the second stage, another HMM model is introduced to train from the generator's output, which boosts the performance.
  arXiv  Detail & Related papers  (2022-07-29T09:29:28Z)
• A transfer learning based approach for pronunciation scoring [7.98890440106366]
  Phone-level pronunciation scoring is a challenging task, with performance far from that of human annotators. Standard systems generate a score for each phone in a phrase using models trained for automatic speech recognition (ASR) with native data only. We present a transfer learning-based approach that leverages a model trained for ASR, adapting it for the task of pronunciation scoring.
  arXiv  Detail & Related papers  (2021-11-01T14:37:06Z)
• LDNet: Unified Listener Dependent Modeling in MOS Prediction for Synthetic Speech [67.88748572167309]
  We present LDNet, a unified framework for mean opinion score (MOS) prediction. We propose two inference methods that provide more stable results and efficient computation.
  arXiv  Detail & Related papers  (2021-10-18T08:52:31Z)
• Dynamic Acoustic Unit Augmentation With BPE-Dropout for Low-Resource End-to-End Speech Recognition [62.94773371761236]
  We consider building an effective end-to-end ASR system in low-resource setups with a high OOV rate. We propose a method of dynamic acoustic unit augmentation based on the BPE-dropout technique. Our monolingual Turkish Conformer established a competitive result with 22.2% character error rate (CER) and 38.9% word error rate (WER)
  arXiv  Detail & Related papers  (2021-03-12T10:10:13Z)
• Statistical Context-Dependent Units Boundary Correction for Corpus-based Unit-Selection Text-to-Speech [1.4337588659482519]
  We present an innovative technique for speaker adaptation in order to improve the accuracy of segmentation with application to unit-selection Text-To-Speech (TTS) systems. Unlike conventional techniques for speaker adaptation, we aim to use only context dependent characteristics extrapolated with linguistic analysis techniques.
  arXiv  Detail & Related papers  (2020-03-05T12:42:13Z)
• Phoneme Boundary Detection using Learnable Segmental Features [31.203969460341817]
  Phoneme boundary detection plays an essential first step for a variety of speech processing applications. We propose a neural architecture coupled with a parameterized structured loss function to learn segmental representations for the task of phoneme boundary detection.
  arXiv  Detail & Related papers  (2020-02-11T14:03:08Z)
• End-to-End Automatic Speech Recognition Integrated With CTC-Based Voice Activity Detection [48.80449801938696]
  This paper integrates a voice activity detection function with end-to-end automatic speech recognition. We focus on connectionist temporal classification ( CTC) and its extension ofsynchronous/attention. We use the labels as a cue for detecting speech segments with simple thresholding.
  arXiv  Detail & Related papers  (2020-02-03T03:36:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.