Seamless Dysfluent Speech Text Alignment for Disordered Speech Analysis

• URL: http://arxiv.org/abs/2506.12073v1
• Date: Thu, 05 Jun 2025 03:06:37 GMT
• Title: Seamless Dysfluent Speech Text Alignment for Disordered Speech Analysis
• Authors: Zongli Ye, Jiachen Lian, Xuanru Zhou, Jinming Zhang, Haodong Li, Shuhe Li, Chenxu Guo, Anaisha Das, Peter Park, Zoe Ezzes, Jet Vonk, Brittany Morin, Rian Bogley, Lisa Wauters, Zachary Miller, Maria Gorno-Tempini, Gopala Anumanchipalli,
• Abstract summary: We propose Neural LCS, a novel approach for dysfluent text-text and speech-text alignment.<n>We evaluate our method on a large-scale simulated dataset.<n>Our results demonstrate the potential of Neural LCS to enhance automated systems for diagnosing and analyzing speech disorders.
• Score: 8.5693791544413
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Accurate alignment of dysfluent speech with intended text is crucial for automating the diagnosis of neurodegenerative speech disorders. Traditional methods often fail to model phoneme similarities effectively, limiting their performance. In this work, we propose Neural LCS, a novel approach for dysfluent text-text and speech-text alignment. Neural LCS addresses key challenges, including partial alignment and context-aware similarity mapping, by leveraging robust phoneme-level modeling. We evaluate our method on a large-scale simulated dataset, generated using advanced data simulation techniques, and real PPA data. Neural LCS significantly outperforms state-of-the-art models in both alignment accuracy and dysfluent speech segmentation. Our results demonstrate the potential of Neural LCS to enhance automated systems for diagnosing and analyzing speech disorders, offering a more accurate and linguistically grounded solution for dysfluent speech alignment.

Related papers

• Analysis and Evaluation of Synthetic Data Generation in Speech Dysfluency Detection [5.95376852691752]
  Speech dysfluency detection is crucial for clinical diagnosis and language assessment.<n>This dataset captures 11 dysfluency categories spanning both word and phoneme levels.<n>Building upon this resource, we improve an end-to-end dysfluency detection framework.
  arXiv  Detail & Related papers  (2025-05-28T06:52:10Z)
• Dysfluent WFST: A Framework for Zero-Shot Speech Dysfluency Transcription and Detection [5.512072120303165]
  Dysfluent-WFST is a zero-shot decoder that simultaneously transcribes phonemes and detects dysfluency.<n>It achieves state-of-the-art performance in both phonetic error rate and dysfluency detection on simulated and real speech data.
  arXiv  Detail & Related papers  (2025-05-22T08:02:50Z)
• Dialogue is Better Than Monologue: Instructing Medical LLMs via Strategical Conversations [74.83732294523402]
  We introduce a novel benchmark that simulates real-world diagnostic scenarios, integrating noise and difficulty levels aligned with USMLE standards.<n>We also explore dialogue-based fine-tuning, which transforms static datasets into conversational formats to better capture iterative reasoning processes.<n>Experiments show that dialogue-tuned models outperform traditional methods, with improvements of $9.64%$ in multi-round reasoning scenarios and $6.18%$ in accuracy in a noisy environment.
  arXiv  Detail & Related papers  (2025-01-29T18:58:48Z)
• Where are we in audio deepfake detection? A systematic analysis over generative and detection models [59.09338266364506]
  SONAR is a synthetic AI-Audio Detection Framework and Benchmark.<n>It provides a comprehensive evaluation for distinguishing cutting-edge AI-synthesized auditory content.<n>It is the first framework to uniformly benchmark AI-audio detection across both traditional and foundation model-based detection systems.
  arXiv  Detail & Related papers  (2024-10-06T01:03:42Z)
• YOLO-Stutter: End-to-end Region-Wise Speech Dysfluency Detection [5.42845980208244]
  YOLO-Stutter is a first end-to-end method that detects dysfluencies in a time-accurate manner. VCTK-Stutter and VCTK-TTS simulate natural spoken dysfluencies including repetition, block, missing, replacement, and prolongation.
  arXiv  Detail & Related papers  (2024-08-27T11:31:12Z)
• Towards Hierarchical Spoken Language Dysfluency Modeling [8.45042473491412]
  Speech disfluency modeling is the bottleneck for both speech therapy and language learning. We present Hierarchical Unconstrained Disfluency Modeling (H-UDM) approach, the hierarchical extension of UDM. Our experimental findings serve as clear evidence of the effectiveness and reliability of the methods we have introduced.
  arXiv  Detail & Related papers  (2024-01-18T14:33:01Z)
• Weakly-supervised forced alignment of disfluent speech using phoneme-level modeling [10.283092375534311]
  We propose a simple and effective modification of alignment graph construction using weighted Finite State Transducers. The proposed weakly-supervised approach alleviates the need for verbatim transcription of speech disfluencies for forced alignment. Our evaluation on a corrupted version of the TIMIT test set and the UCLASS dataset shows significant improvements.
  arXiv  Detail & Related papers  (2023-05-30T09:57:36Z)
• Discretization and Re-synthesis: an alternative method to solve the Cocktail Party Problem [65.25725367771075]
  This study demonstrates, for the first time, that the synthesis-based approach can also perform well on this problem. Specifically, we propose a novel speech separation/enhancement model based on the recognition of discrete symbols. By utilizing the synthesis model with the input of discrete symbols, after the prediction of discrete symbol sequence, each target speech could be re-synthesized.
  arXiv  Detail & Related papers  (2021-12-17T08:35:40Z)
• Factorized Neural Transducer for Efficient Language Model Adaptation [51.81097243306204]
  We propose a novel model, factorized neural Transducer, by factorizing the blank and vocabulary prediction. It is expected that this factorization can transfer the improvement of the standalone language model to the Transducer for speech recognition. We demonstrate that the proposed factorized neural Transducer yields 15% to 20% WER improvements when out-of-domain text data is used for language model adaptation.
  arXiv  Detail & Related papers  (2021-09-27T15:04:00Z)
• Multi-Modal Detection of Alzheimer's Disease from Speech and Text [3.702631194466718]
  We propose a deep learning method that utilizes speech and the corresponding transcript simultaneously to detect Alzheimer's disease (AD) The proposed method achieves 85.3% 10-fold cross-validation accuracy when trained and evaluated on the Dementiabank Pitt corpus.
  arXiv  Detail & Related papers  (2020-11-30T21:18:17Z)
• Neural Data-to-Text Generation via Jointly Learning the Segmentation and Correspondence [48.765579605145454]
  We propose to explicitly segment target text into fragment units and align them with their data correspondences. The resulting architecture maintains the same expressive power as neural attention models. On both E2E and WebNLG benchmarks, we show the proposed model consistently outperforms its neural attention counterparts.
  arXiv  Detail & Related papers  (2020-05-03T14:28:28Z)
• Data Augmentation for Spoken Language Understanding via Pretrained Language Models [113.56329266325902]
  Training of spoken language understanding (SLU) models often faces the problem of data scarcity. We put forward a data augmentation method using pretrained language models to boost the variability and accuracy of generated utterances.
  arXiv  Detail & Related papers  (2020-04-29T04:07:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.