Automated Pronunciation Evaluation for Korean Toddler Speech using Speech Diarization and Self-Supervised Learning

• URL: http://arxiv.org/abs/2606.10213v1
• Date: Mon, 08 Jun 2026 22:07:59 GMT
• Title: Automated Pronunciation Evaluation for Korean Toddler Speech using Speech Diarization and Self-Supervised Learning
• Abstract summary: Speech sound disorders affect approximately 44% of Korean pediatric communication disorder cases.<n>This paper presents an end-to-end pipeline for automated pronunciation evaluation of Korean toddler speech.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Speech sound disorders affect approximately 44% of Korean pediatric communication disorder cases, yet automated assessment tools for Korean toddler speech remain underdeveloped. This paper presents an end-to-end pipeline for automated pronunciation evaluation of Korean toddler speech, combining neural speaker diarization with self-supervised speech representation learning. We introduce a novel IRB-approved corpus of 53 recordings from Korean-speaking children aged 2-5 years. A subset of 53 subjects was annotated by three independent reviewers, yielding 1,190 consonant and 748 vowel word-level binary correctness labels. We evaluate three diarization models, finding that NeMo SortFormer achieves 88.69% speaker count accuracy and 33.04% diarization error rate (DER) owing to its arrival-time-sorted transformer architecture, which handles the acoustic confound between young female caregivers exhibiting aegyo and toddler speech. For pronunciation scoring, we compare three self-supervised learning (SSL) backbones across multiple pooling strategies. A cross-model ensemble routing consonant prediction to HuBERT-large and vowel prediction to WavLM-large achieves balanced accuracies of 0.720 and 0.845, with a mean of 0.782.

Related papers

• S-DiverSe: Spanish Diverse Speech [67.47251829943015]
  S-DiverSe is a corpus of 3.2 hours of in-the-wild Spanish speech from 22 speakers with amyotrophic lateral sclerosis, Parkinson's disease, and stroke.<n>The dataset contains 444 manually transcribed audio segments with metadata on speaker sex, disease type, and intelligibility.
  arXiv  Detail & Related papers  (2026-07-03T11:23:43Z)
• KVoiceBench, KOpenAudioBench, and KMMAU: Agent-Driven Korean Speech Benchmarks for Evaluating SpeechLMs [7.195547595036643]
  We propose two human-agent benchmark-construction frameworks for SpeechLM evaluation.<n>One transfers source-language SpokenQA benchmarks into target-language SpokenQA benchmarks, and the other converts target-language ASR corpora into audio understanding benchmarks.<n>Using these frameworks, we construct and publicly release three Korean speech benchmarks: KVoiceBench and KOpenAudioBench for Korean SpokenQA, and KMMAU for Korean audio understanding, comprising 12,345 samples in total.
  arXiv  Detail & Related papers  (2026-05-27T05:19:38Z)
• Harf-Speech: A Clinically Aligned Framework for Arabic Phoneme-Level Speech Assessment [1.0898519359579681]
  Harf-Speech is a modular system scoring Arabic pronunciation at the phoneme level on a clinical scale.<n>Harf-Speech attains a Pearson correlation of 0.791 and ICC (2,1) of 0.659 with mean expert scores, outperforming existing end-to-end assessment frameworks.
  arXiv  Detail & Related papers  (2026-03-11T04:10:35Z)
• DialectGen: Benchmarking and Improving Dialect Robustness in Multimodal Generation [111.94720088481614]
  Can multimodal generative models effectively produce content given dialectal textual input?<n>We construct a new large-scale benchmark spanning six common English dialects.<n>We design a general encoder-based mitigation strategy for multimodal generative models.
  arXiv  Detail & Related papers  (2025-10-16T17:56:55Z)
• Speech Corpus for Korean Children with Autism Spectrum Disorder: Towards Automatic Assessment Systems [7.153773998764661]
  This paper introduces a speech corpus specifically designed for Korean children with ASD. Three speech and language pathologists rated recordings for social communication severity (SCS) and pronunciation proficiency (PP) using a 3-point Likert scale. The paper also analyzes acoustic and linguistic features extracted from speech data collected and completed for annotation from 73 children with ASD and 9 TD children.
  arXiv  Detail & Related papers  (2024-02-23T07:32:54Z)
• Who Said What? An Automated Approach to Analyzing Speech in Preschool Classrooms [0.4207829324073153]
  We propose an automated framework that uses software to classify speakers and to transcribe their utterances. We compare results from our framework to those from a human expert for 110 minutes of classroom recordings. The results suggest substantial progress in analyzing classroom speech that may support children's language development.
  arXiv  Detail & Related papers  (2024-01-14T18:27:37Z)
• Decoding speech perception from non-invasive brain recordings [48.46819575538446]
  We introduce a model trained with contrastive-learning to decode self-supervised representations of perceived speech from non-invasive recordings. Our model can identify, from 3 seconds of MEG signals, the corresponding speech segment with up to 41% accuracy out of more than 1,000 distinct possibilities.
  arXiv  Detail & Related papers  (2022-08-25T10:01:43Z)
• Exploiting Cross-domain And Cross-Lingual Ultrasound Tongue Imaging Features For Elderly And Dysarthric Speech Recognition [55.25565305101314]
  Articulatory features are invariant to acoustic signal distortion and have been successfully incorporated into automatic speech recognition systems. This paper presents a cross-domain and cross-lingual A2A inversion approach that utilizes the parallel audio and ultrasound tongue imaging (UTI) data of the 24-hour TaL corpus in A2A model pre-training. Experiments conducted on three tasks suggested incorporating the generated articulatory features consistently outperformed the baseline TDNN and Conformer ASR systems.
  arXiv  Detail & Related papers  (2022-06-15T07:20:28Z)
• Experiments of ASR-based mispronunciation detection for children and adult English learners [7.083737676329174]
  We develop a mispronunciation assessment system that checks the pronunciation of non-native English speakers. We present an evaluation of the non-native pronunciation observed in phonetically annotated speech corpora.
  arXiv  Detail & Related papers  (2021-04-13T07:24:05Z)
• UniSpeech: Unified Speech Representation Learning with Labeled and Unlabeled Data [54.733889961024445]
  We propose a unified pre-training approach called UniSpeech to learn speech representations with both unlabeled and labeled data. We evaluate the effectiveness of UniSpeech for cross-lingual representation learning on public CommonVoice corpus.
  arXiv  Detail & Related papers  (2021-01-19T12:53:43Z)
• Continuous Speech Separation with Conformer [60.938212082732775]
  We use transformer and conformer in lieu of recurrent neural networks in the separation system. We believe capturing global information with the self-attention based method is crucial for the speech separation.
  arXiv  Detail & Related papers  (2020-08-13T09:36:05Z)
• Unsupervised Cross-lingual Representation Learning for Speech Recognition [63.85924123692923]
  XLSR learns cross-lingual speech representations by pretraining a single model from the raw waveform of speech in multiple languages. We build on wav2vec 2.0 which is trained by solving a contrastive task over masked latent speech representations. Experiments show that cross-lingual pretraining significantly outperforms monolingual pretraining.
  arXiv  Detail & Related papers  (2020-06-24T18:25:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.