Elderly-Contextual Data Augmentation via Speech Synthesis for Elderly ASR

• URL: http://arxiv.org/abs/2604.24770v1
• Date: Wed, 15 Apr 2026 12:56:11 GMT
• Title: Elderly-Contextual Data Augmentation via Speech Synthesis for Elderly ASR
• Abstract summary: Elderly ASR (EASR) remains challenging due to limited training data and the distinct acoustic and linguistic characteristics of elderly speech.<n>We propose a data augmentation pipeline that combines large language model (LLM)-based transcript paraphrasing with text-to-speech synthesis.<n> Experiments on English and Korean elderly speech datasets from speakers aged 70 and above show that the proposed method consistently improves performance.
• Score: 5.231909048842117
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Despite recent progress in automatic speech recognition (ASR), elderly ASR (EASR) remains challenging due to limited training data and the distinct acoustic and linguistic characteristics of elderly speech. In this work, we address data scarcity in EASR through a data augmentation pipeline that combines large language model (LLM)-based transcript paraphrasing with text-to-speech (TTS) synthesis. Given an elderly speech dataset, the LLM first generates elderly-contextual paraphrases of the original transcripts, and the TTS model then synthesizes corresponding speech using elderly reference speakers. The resulting synthetic audio-text pairs are merged with the original data to fine-tune Whisper without architectural modification. We further analyze the effects of augmentation ratio and reference-speaker composition in low-resource EASR. Experiments on English and Korean elderly speech datasets from speakers aged 70 and above show that the proposed method consistently improves performance over conventional augmentation baselines, achieving up to a 58.2% reduction in word error rate (WER) compared with the Whisper baseline.

Related papers

• Rethinking Speech-LLM Integration for ASR: Effective Joint Speech-Text Training by Interleaving [54.66679099886108]
  Speech-LLM integration has shown promising results by leveraging extensive textual pretraining, yet its specific benefits for automatic speech recognition (ASR) remain unclear.<n>We propose Joint Speech-Text Interleaved Pretraining (JSTIP), an ASR-oriented pretraining strategy that constructs word-level and segment-level interleaved speech-text sequences.<n> Experiments on 38k hours of ASR data show consistent entity accuracy improvement compared to ASR-only and joint speech-text training baselines.
  arXiv  Detail & Related papers  (2026-07-02T05:42:01Z)
• Imitation Learning for Elder-Facing Speech Synthesis [65.98429501808018]
  We propose a novel imitation learning (IL) framework to learn text-to-speech (TTS) models from expert demonstrations.<n>We further improve Group Relative Policy Optimization with two-stage on-policy reward learning (OPRL) to mitigate reward hacking under limited supervision from expert demonstration.
  arXiv  Detail & Related papers  (2026-06-19T02:36:35Z)
• Improving Code-Switching Speech Recognition with TTS Data Augmentation [58.34842693152991]
  This paper explores multilingual text-to-speech (TTS) models as an effective data augmentation technique to address this shortage.<n>We fine-tune the multilingual CosyVoice2 TTS model on the SEAME dataset to generate synthetic conversational Chinese-English code-switching speech.
  arXiv  Detail & Related papers  (2026-01-02T10:11:51Z)
• Adapting Foundation Speech Recognition Models to Impaired Speech: A Semantic Re-chaining Approach for Personalization of German Speech [0.562479170374811]
  Speech impairments caused by conditions such as cerebral palsy or genetic disorders pose significant challenges for automatic speech recognition systems.<n>We propose a practical and lightweight pipeline to personalize ASR models, formalizing the selection of words and enriching a small, speech-impaired dataset with semantic coherence.<n>Our approach shows promising improvements in transcription quality, demonstrating the potential to reduce communication barriers for individuals with atypical speech patterns.
  arXiv  Detail & Related papers  (2025-06-23T15:30:50Z)
• An Exhaustive Evaluation of TTS- and VC-based Data Augmentation for ASR [12.197936305117407]
  Augmenting the training data of automatic speech recognition systems with synthetic data generated by text-to-speech (TTS) or voice conversion (VC) has gained popularity in recent years.<n>We leverage recently proposed flow-based TTS/VC models allowing greater speech diversity, and assess the respective impact of augmenting various speech attributes on the word error rate (WER) achieved by several ASR models.
  arXiv  Detail & Related papers  (2025-03-11T23:09:06Z)
• Self-supervised ASR Models and Features For Dysarthric and Elderly Speech Recognition [71.87998918300806]
  This paper explores approaches to integrate domain fine-tuned SSL pre-trained models and their features into TDNN and Conformer ASR systems. TDNN systems constructed by integrating domain-adapted HuBERT, wav2vec2-conformer or multi-lingual XLSR models consistently outperform standalone fine-tuned SSL pre-trained models. Consistent improvements in Alzheimer's Disease detection accuracy are also obtained using the DementiaBank Pitt elderly speech recognition outputs.
  arXiv  Detail & Related papers  (2024-07-03T08:33:39Z)
• AS-70: A Mandarin stuttered speech dataset for automatic speech recognition and stuttering event detection [46.855958156126164]
  AS-70 is the first publicly available Mandarin stuttered speech dataset. This paper introduces AS-70, the first publicly available Mandarin stuttered speech dataset.
  arXiv  Detail & Related papers  (2024-06-11T13:35:50Z)
• Hyper-parameter Adaptation of Conformer ASR Systems for Elderly and Dysarthric Speech Recognition [64.9816313630768]
  Fine-tuning is often used to exploit the large quantities of non-aged and healthy speech pre-trained models. This paper investigates hyper- parameter adaptation for Conformer ASR systems that are pre-trained on the Librispeech corpus.
  arXiv  Detail & Related papers  (2023-06-27T07:49:35Z)
• Personalized Adversarial Data Augmentation for Dysarthric and Elderly Speech Recognition [30.885165674448352]
  This paper presents a novel set of speaker dependent (GAN) based data augmentation approaches for elderly and dysarthric speech recognition. GAN based data augmentation approaches consistently outperform the baseline speed perturbation method by up to 0.91% and 3.0% absolute. Consistent performance improvements are retained after applying LHUC based speaker adaptation.
  arXiv  Detail & Related papers  (2022-05-13T04:29:49Z)
• On-the-Fly Feature Based Rapid Speaker Adaptation for Dysarthric and Elderly Speech Recognition [53.17176024917725]
  Scarcity of speaker-level data limits the practical use of data-intensive model based speaker adaptation methods. This paper proposes two novel forms of data-efficient, feature-based on-the-fly speaker adaptation methods.
  arXiv  Detail & Related papers  (2022-03-28T09:12:24Z)
• Data augmentation using prosody and false starts to recognize non-native children's speech [12.911954427107977]
  This paper describes AaltoASR's speech recognition system for the INTERSPEECH 2020 shared task on Automatic Speech Recognition. The task is to recognize non-native speech from children of various age groups given a limited amount of speech.
  arXiv  Detail & Related papers  (2020-08-29T05:32:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.