K-Function: Joint Pronunciation Transcription and Feedback for Evaluating Kids Language Function

• URL: http://arxiv.org/abs/2507.03043v1
• Date: Thu, 03 Jul 2025 08:05:02 GMT
• Title: K-Function: Joint Pronunciation Transcription and Feedback for Evaluating Kids Language Function
• Authors: Shuhe Li, Chenxu Guo, Jiachen Lian, Cheol Jun Cho, Wenshuo Zhao, Xuanru Zhou, Dingkun Zhou, Sam Wang, Grace Wang, Jingze Yang, Jingyi Xu, Ruohan Bao, Elise Brenner, Brandon In, Francesca Pei, Maria Luisa Gorno-Tempini, Gopala Anumanchipalli,
• Abstract summary: We introduce K-Function, a unified framework that combines accurate sub-word transcription, objective scoring, and actionable feedback.<n>Kids-WFST attains 1.39% phoneme error on MyST and 8.61% on Multitudes--absolute gains of 10.47 and 7.06 points over a greedy-search decoder.
• Score: 10.918072285423706
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Early evaluation of children's language is frustrated by the high pitch, long phones, and sparse data that derail automatic speech recognisers. We introduce K-Function, a unified framework that combines accurate sub-word transcription, objective scoring, and actionable feedback. Its core, Kids-WFST, merges a Wav2Vec2 phoneme encoder with a phoneme-similarity Dysfluent-WFST to capture child-specific errors while remaining fully interpretable. Kids-WFST attains 1.39% phoneme error on MyST and 8.61% on Multitudes--absolute gains of 10.47 and 7.06 points over a greedy-search decoder. These high-fidelity transcripts power an LLM that grades verbal skills, milestones, reading, and comprehension, aligning with human proctors and supplying tongue-and-lip visualizations plus targeted advice. The results show that precise phoneme recognition cements a complete diagnostic-feedback loop, paving the way for scalable, clinician-ready language assessment.

Related papers

• Improving Child Speech Recognition and Reading Mistake Detection by Using Prompts [10.137389745562512]
  Best performing system achieved state-of-the-art recognition performance in Dutch child read speech.<n>It significantly improved reading mistake detection, increasing the F1 score from 0.39 to 0.73.
  arXiv  Detail & Related papers  (2025-06-04T05:55:12Z)
• Automated evaluation of children's speech fluency for low-resource languages [8.918459083715149]
  This paper proposes a system to automatically assess fluency by combining a fine-tuned multilingual ASR model and an objective metrics extraction stage.<n>We evaluate the proposed system on a dataset of children's speech in two low-resource languages, Tamil and Malay.
  arXiv  Detail & Related papers  (2025-05-26T08:25:50Z)
• Languages in Multilingual Speech Foundation Models Align Both Phonetically and Semantically [58.019484208091534]
  Cross-lingual alignment in pretrained language models (LMs) has enabled efficient transfer in text-based LMs.<n>It remains an open question whether findings and methods from text-based cross-lingual alignment apply to speech.
  arXiv  Detail & Related papers  (2025-05-26T07:21:20Z)
• Can Prompting LLMs Unlock Hate Speech Detection across Languages? A Zero-shot and Few-shot Study [59.30098850050971]
  This work evaluates LLM prompting-based detection across eight non-English languages.<n>We show that while zero-shot and few-shot prompting lag behind fine-tuned encoder models on most of the real-world evaluation sets, they achieve better generalization on functional tests for hate speech detection.
  arXiv  Detail & Related papers  (2025-05-09T16:00:01Z)
• CLAIR-A: Leveraging Large Language Models to Judge Audio Captions [73.51087998971418]
  evaluating machine-generated audio captions is a complex task that requires considering diverse factors. We propose CLAIR-A, a simple and flexible method that leverages the zero-shot capabilities of large language models. In our evaluations, CLAIR-A better predicts human judgements of quality compared to traditional metrics.
  arXiv  Detail & Related papers  (2024-09-19T17:59:52Z)
• Exploring the Factual Consistency in Dialogue Comprehension of Large Language Models [51.75805497456226]
  This work focuses on the factual consistency issue with the help of the dialogue summarization task. Our evaluation shows that, on average, 26.8% of the summaries generated by LLMs contain factual inconsistency. To stimulate and enhance the dialogue comprehension ability of LLMs, we propose a fine-tuning paradigm with auto-constructed multi-task data.
  arXiv  Detail & Related papers  (2023-11-13T09:32:12Z)
• Using Natural Language Explanations to Rescale Human Judgments [81.66697572357477]
  We propose a method to rescale ordinal annotations and explanations using large language models (LLMs)<n>We feed annotators' Likert ratings and corresponding explanations into an LLM and prompt it to produce a numeric score anchored in a scoring rubric.<n>Our method rescales the raw judgments without impacting agreement and brings the scores closer to human judgments grounded in the same scoring rubric.
  arXiv  Detail & Related papers  (2023-05-24T06:19:14Z)
• On the Off-Target Problem of Zero-Shot Multilingual Neural Machine Translation [104.85258654917297]
  We find that failing in encoding discriminative target language signal will lead to off-target and a closer lexical distance. We propose Language Aware Vocabulary Sharing (LAVS) to construct the multilingual vocabulary. We conduct experiments on a multilingual machine translation benchmark in 11 languages.
  arXiv  Detail & Related papers  (2023-05-18T12:43:31Z)
• Nonwords Pronunciation Classification in Language Development Tests for Preschool Children [7.224391516694955]
  This work aims to automatically evaluate whether the language development of children is age-appropriate. In this work, the task is to determine whether spoken nonwords have been uttered correctly. We compare different approaches that are motivated to model specific language structures.
  arXiv  Detail & Related papers  (2022-06-16T10:19:47Z)
• An Approach to Mispronunciation Detection and Diagnosis with Acoustic, Phonetic and Linguistic (APL) Embeddings [18.282632348274756]
  Phonetic embeddings, extracted from ASR models trained with huge amount of word level annotations, can serve as a good representation of the content of input speech. We propose to utilize Acoustic, Phonetic and Linguistic (APL) embedding features jointly for building a more powerful MD&D system.
  arXiv  Detail & Related papers  (2021-10-14T11:25:02Z)
• UniSpeech: Unified Speech Representation Learning with Labeled and Unlabeled Data [54.733889961024445]
  We propose a unified pre-training approach called UniSpeech to learn speech representations with both unlabeled and labeled data. We evaluate the effectiveness of UniSpeech for cross-lingual representation learning on public CommonVoice corpus.
  arXiv  Detail & Related papers  (2021-01-19T12:53:43Z)
• Data augmentation using prosody and false starts to recognize non-native children's speech [12.911954427107977]
  This paper describes AaltoASR's speech recognition system for the INTERSPEECH 2020 shared task on Automatic Speech Recognition. The task is to recognize non-native speech from children of various age groups given a limited amount of speech.
  arXiv  Detail & Related papers  (2020-08-29T05:32:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.