Reasoning Beyond Transcription: Audio Language Models on Child Stuttering Speech

• URL: http://arxiv.org/abs/2609.07968v1
• Date: Mon, 07 Sep 2026 20:44:38 GMT
• Title: Reasoning Beyond Transcription: Audio Language Models on Child Stuttering Speech
• Abstract summary: Child speech differs from adult speech in acoustics, prosody, and linguistic structures.<n>While Audio Language Models (ALMs) show strong semantic reasoning from speech audio, their ability to reason about disfluent child speech in mixed-speaker settings remains unexplored.<n>We investigate this through two tasks: child-focused semantic summarization and speech entailment.
• Score: 9.596167964492588
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Child speech differs from adult speech in acoustics, prosody, and linguistic structures. Speech disfluencies (such as repetitions) further challenge automatic understanding. While Audio Language Models (ALMs) show strong semantic reasoning from speech audio, their ability to reason about disfluent child speech in mixed-speaker settings remains unexplored. We investigate this through two tasks: child-focused semantic summarization and speech entailment. Experiments use recordings of children who stutter in mixed speaker interviews without explicit speaker separation. Models are instruction-guided to focus on the child, preserve clinically relevant disfluencies, and avoid adult-speech leakage. Evaluation combines LLM-based judges and reference-based metrics, anchored by transcript-oracle baselines to isolate errors. Results show that while ALMs extract high-level meaning from stuttered speech, reasoning degrades significantly with increased

Related papers

• Speaker-Disentangled Chunk-Wise Regression for Syllabic Tokenization [14.571581599991331]
  Unsupervised syllabic tokenization aims to learn discrete syllabic tokens that capture latent linguistic content-related structure from raw speech.<n>Recent syllabic tokenization methods employ teacher-student distillation of the pretrained HuBERT to organize latent speech frame representations into syllabic segments.<n>We propose a speaker-disentangled syllabic tokenizer that regresses speaker-perturbed student representations toward clean teacher targets within fixed-length chunks.
  arXiv  Detail & Related papers  (2026-07-05T00:39:57Z)
• Can large audio language models understand child stuttering speech? speech summarization, and source separation [3.2684800403907506]
  Child speech differs from adult speech in acoustics, prosody, and language development, and disfluencies (repetitions, prolongations, blocks)<n>Recent large audio-language models (LALMs) demonstrate strong cross-modal audio understanding.<n>We evaluate several state-of-the-art LALMs in two settings: an interview (mixed speakers) and a reading task (single child)
  arXiv  Detail & Related papers  (2025-10-21T18:53:34Z)
• SpeechR: A Benchmark for Speech Reasoning in Large Audio-Language Models [60.72029578488467]
  SpeechR is a unified benchmark for evaluating reasoning over speech in large audio-language models.<n>It evaluates models along three key dimensions: factual retrieval, procedural inference, and normative judgment.<n> Evaluations on eleven state-of-the-art LALMs reveal that high transcription accuracy does not translate into strong reasoning capabilities.
  arXiv  Detail & Related papers  (2025-08-04T03:28:04Z)
• ProsodyLM: Uncovering the Emerging Prosody Processing Capabilities in Speech Language Models [70.56468982313834]
  We propose ProsodyLM, which introduces a simple tokenization scheme amenable to learning prosody.<n>We find that ProsodyLM can learn surprisingly diverse emerging prosody processing capabilities through pre-training alone.
  arXiv  Detail & Related papers  (2025-07-27T00:59:01Z)
• Self-supervised Speech Models for Word-Level Stuttered Speech Detection [66.46810024006712]
  We introduce a word-level stuttering speech detection model leveraging self-supervised speech models. Our evaluation demonstrates that our model surpasses previous approaches in word-level stuttering speech detection.
  arXiv  Detail & Related papers  (2024-09-16T20:18:20Z)
• MixSpeech: Cross-Modality Self-Learning with Audio-Visual Stream Mixup for Visual Speech Translation and Recognition [51.412413996510814]
  We propose MixSpeech, a cross-modality self-learning framework that utilizes audio speech to regularize the training of visual speech tasks. MixSpeech enhances speech translation in noisy environments, improving BLEU scores for four languages on AVMuST-TED by +1.4 to +4.2.
  arXiv  Detail & Related papers  (2023-03-09T14:58:29Z)
• Time out of Mind: Generating Rate of Speech conditioned on emotion and speaker [0.0]
  We train a GAN conditioned on emotion to generate worth lengths for a given input text. These word lengths are relative neutral speech and can be provided to a text-to-speech system to generate more expressive speech. We were able to achieve better performances on objective measures for neutral speech, and better time alignment for happy speech when compared to an out-of-box model.
  arXiv  Detail & Related papers  (2023-01-29T02:58:01Z)
• Improving Self-Supervised Speech Representations by Disentangling Speakers [56.486084431528695]
  Self-supervised learning in speech involves training a speech representation network on a large-scale unannotated speech corpus. Disentangling speakers is very challenging, because removing the speaker information could easily result in a loss of content as well. We propose a new SSL method that can achieve speaker disentanglement without severe loss of content.
  arXiv  Detail & Related papers  (2022-04-20T04:56:14Z)
• Audio-Visual Speech Codecs: Rethinking Audio-Visual Speech Enhancement by Re-Synthesis [67.73554826428762]
  We propose a novel audio-visual speech enhancement framework for high-fidelity telecommunications in AR/VR. Our approach leverages audio-visual speech cues to generate the codes of a neural speech, enabling efficient synthesis of clean, realistic speech from noisy signals.
  arXiv  Detail & Related papers  (2022-03-31T17:57:10Z)
• Comparing Supervised Models And Learned Speech Representations For Classifying Intelligibility Of Disordered Speech On Selected Phrases [11.3463024120429]
  We develop and compare different deep learning techniques to classify the intelligibility of disordered speech on selected phrases. We collected samples from a diverse set of 661 speakers with a variety of self-reported disorders speaking 29 words or phrases.
  arXiv  Detail & Related papers  (2021-07-08T17:24:25Z)
• Learning to Understand Child-directed and Adult-directed Speech [18.29692441616062]
  Human language acquisition research indicates that child-directed speech helps language learners. We compare the task performance of models trained on adult-directed speech (ADS) and child-directed speech (CDS) We find indications that CDS helps in the initial stages of learning, but eventually, models trained on ADS reach comparable task performance, and generalize better.
  arXiv  Detail & Related papers  (2020-05-06T10:47:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.