Adapting Foundation ASR Models to Dysarthric Speech: A Case Study

• URL: http://arxiv.org/abs/2606.31722v1
• Date: Tue, 30 Jun 2026 14:23:49 GMT
• Title: Adapting Foundation ASR Models to Dysarthric Speech: A Case Study
• Abstract summary: This paper presents a personalized ASR system for a dysarthric speaker built by adapting a foundation ASR model to speaker-specific data.<n>Using the TEQST tool, we collected 92 hours of read speech and later added 8.8 hours of user corrections gathered through a deployed mobile application.
• Score: 56.38963587230989
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Automatic speech recognition (ASR) systems often perform poorly in dysarthric speech, limiting their usefulness to affected speakers in everyday communication. This paper presents a personalized ASR system for a dysarthric speaker, built by adapting a foundation ASR model to speaker-specific data. Using the TEQST tool, we collected 92 hours of read speech and later added 8.8 hours of user corrections gathered through a deployed mobile application. Starting from Whisper, fine-tuning reduced word error rate to 15.8% with only 1.4 hours of adaptation data, reached 10.7% with 22.5 hours, and achieved the best result of 9.7% when using all available data including the corrections. Using LoRA adaptation and/or Qwen3-ASR as foundation model performed worse in this setting. The results show that personalized fine-tuning can make foundation ASR models substantially more effective for dysarthric speech and suitable for practical deployment.

Related papers

• Towards Pretraining Robust ASR Foundation Model with Acoustic-Aware Data Augmentation [18.678742816040856]
  Whisper's robust performance in automatic speech recognition (ASR) is often attributed to its massive 680k-hour training set.<n>We examine how linguistic and acoustic diversity in training data affect the robustness of the ASR model.<n>We find that targeted acoustic augmentation methods could significantly improve the generalization ability of ASR models.
  arXiv  Detail & Related papers  (2025-05-27T00:55:32Z)
• Enhancing AAC Software for Dysarthric Speakers in e-Health Settings: An Evaluation Using TORGO [0.13108652488669734]
  Individuals with cerebral palsy (CP) and amyotrophic lateral sclerosis (ALS) frequently face challenges with articulation, leading to dysarthria and resulting in atypical speech patterns. We found that state-of-the-art (SOTA) automatic speech recognition (ASR) technology like Whisper and Wav2vec2.0 marginalizes atypical speakers largely due to the lack of training data. Our work looks to leverage SOTA ASR followed by domain specific error-correction.
  arXiv  Detail & Related papers  (2024-11-01T19:11:54Z)
• Homogeneous Speaker Features for On-the-Fly Dysarthric and Elderly Speaker Adaptation [71.31331402404662]
  This paper proposes two novel data-efficient methods to learn dysarthric and elderly speaker-level features. Speaker-regularized spectral basis embedding-SBE features that exploit a special regularization term to enforce homogeneity of speaker features in adaptation. Feature-based learning hidden unit contributions (f-LHUC) that are conditioned on VR-LH features that are shown to be insensitive to speaker-level data quantity in testtime adaptation.
  arXiv  Detail & Related papers  (2024-07-08T18:20:24Z)
• Self-supervised ASR Models and Features For Dysarthric and Elderly Speech Recognition [71.87998918300806]
  This paper explores approaches to integrate domain fine-tuned SSL pre-trained models and their features into TDNN and Conformer ASR systems. TDNN systems constructed by integrating domain-adapted HuBERT, wav2vec2-conformer or multi-lingual XLSR models consistently outperform standalone fine-tuned SSL pre-trained models. Consistent improvements in Alzheimer's Disease detection accuracy are also obtained using the DementiaBank Pitt elderly speech recognition outputs.
  arXiv  Detail & Related papers  (2024-07-03T08:33:39Z)
• Convoifilter: A case study of doing cocktail party speech recognition [59.80042864360884]
  The model can decrease ASR's word error rate (WER) from 80% to 26.4% through this approach. We openly share our pre-trained model to foster further research hf.co/nguyenvulebinh/voice-filter.
  arXiv  Detail & Related papers  (2023-08-22T12:09:30Z)
• Arabic Dysarthric Speech Recognition Using Adversarial and Signal-Based Augmentation [4.874780144224057]
  We aim to improve the performance of Arabic dysarthric automatic speech recognition through a multi-stage augmentation approach. To this effect, we first propose a signal-based approach to generate dysarthric Arabic speech from healthy Arabic speech. We also propose a second stage Parallel Wave Generative (PWG) adversarial model that is trained on an English dysarthric dataset.
  arXiv  Detail & Related papers  (2023-06-07T12:01:46Z)
• Personalized Adversarial Data Augmentation for Dysarthric and Elderly Speech Recognition [30.885165674448352]
  This paper presents a novel set of speaker dependent (GAN) based data augmentation approaches for elderly and dysarthric speech recognition. GAN based data augmentation approaches consistently outperform the baseline speed perturbation method by up to 0.91% and 3.0% absolute. Consistent performance improvements are retained after applying LHUC based speaker adaptation.
  arXiv  Detail & Related papers  (2022-05-13T04:29:49Z)
• On-the-Fly Feature Based Rapid Speaker Adaptation for Dysarthric and Elderly Speech Recognition [53.17176024917725]
  Scarcity of speaker-level data limits the practical use of data-intensive model based speaker adaptation methods. This paper proposes two novel forms of data-efficient, feature-based on-the-fly speaker adaptation methods.
  arXiv  Detail & Related papers  (2022-03-28T09:12:24Z)
• Exploiting Cross Domain Acoustic-to-articulatory Inverted Features For Disordered Speech Recognition [57.15942628305797]
  Articulatory features are invariant to acoustic signal distortion and have been successfully incorporated into automatic speech recognition systems for normal speech. This paper presents a cross-domain acoustic-to-articulatory (A2A) inversion approach that utilizes the parallel acoustic-articulatory data of the 15-hour TORGO corpus in model training. Cross-domain adapted to the 102.7-hour UASpeech corpus and to produce articulatory features.
  arXiv  Detail & Related papers  (2022-03-19T08:47:18Z)
• Investigation of Data Augmentation Techniques for Disordered Speech Recognition [69.50670302435174]
  This paper investigates a set of data augmentation techniques for disordered speech recognition. Both normal and disordered speech were exploited in the augmentation process. The final speaker adapted system constructed using the UASpeech corpus and the best augmentation approach based on speed perturbation produced up to 2.92% absolute word error rate (WER)
  arXiv  Detail & Related papers  (2022-01-14T17:09:22Z)
• Robust Self-Supervised Audio-Visual Speech Recognition [29.526786921769613]
  We present a self-supervised audio-visual speech recognition framework built upon Audio-Visual HuBERT (AV-HuBERT) On the largest available AVSR benchmark dataset LRS3, our approach outperforms prior state-of-the-art by 50% (28.0% vs. 14.1%) using less than 10% of labeled data. Our approach reduces the WER of an audio-based model by over 75% (25.8% vs. 5.8%) on average.
  arXiv  Detail & Related papers  (2022-01-05T18:50:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.