Can Layer-wise SSL Features Improve Zero-Shot ASR Performance for Children's Speech?

• URL: http://arxiv.org/abs/2508.21225v1
• Date: Thu, 28 Aug 2025 21:32:36 GMT
• Title: Can Layer-wise SSL Features Improve Zero-Shot ASR Performance for Children's Speech?
• Authors: Abhijit Sinha, Hemant Kumar Kathania, Sudarsana Reddy Kadiri, Shrikanth Narayanan,
• Abstract summary: This study investigates the effectiveness of layer-wise features extracted from state-of-the-art SSL pre-trained models in improving the performance of ASR for children's speech in zero-shot scenarios.<n>The analysis identified the most effective layers for enhancing ASR performance on children's speech in a zero-shot scenario.
• Score: 43.31597557333867
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Automatic Speech Recognition (ASR) systems often struggle to accurately process children's speech due to its distinct and highly variable acoustic and linguistic characteristics. While recent advancements in self-supervised learning (SSL) models have greatly enhanced the transcription of adult speech, accurately transcribing children's speech remains a significant challenge. This study investigates the effectiveness of layer-wise features extracted from state-of-the-art SSL pre-trained models - specifically, Wav2Vec2, HuBERT, Data2Vec, and WavLM in improving the performance of ASR for children's speech in zero-shot scenarios. A detailed analysis of features extracted from these models was conducted, integrating them into a simplified DNN-based ASR system using the Kaldi toolkit. The analysis identified the most effective layers for enhancing ASR performance on children's speech in a zero-shot scenario, where WSJCAM0 adult speech was used for training and PFSTAR children speech for testing. Experimental results indicated that Layer 22 of the Wav2Vec2 model achieved the lowest Word Error Rate (WER) of 5.15%, representing a 51.64% relative improvement over the direct zero-shot decoding using Wav2Vec2 (WER of 10.65%). Additionally, age group-wise analysis demonstrated consistent performance improvements with increasing age, along with significant gains observed even in younger age groups using the SSL features. Further experiments on the CMU Kids dataset confirmed similar trends, highlighting the generalizability of the proposed approach.

Related papers

• Zero-Shot KWS for Children's Speech using Layer-Wise Features from SSL Models [12.976537689930373]
  This paper introduces a zero-shot KWS approach that leverages state-of-the-art self-supervised learning (SSL) models.<n>Features are extracted layer-wise from these SSL models and used to train a Kaldi-based KWS system.<n>Our approach achieved state-of-the-art results across all keyword sets for children's speech.
  arXiv  Detail & Related papers  (2025-08-28T22:32:42Z)
• Benchmarking Training Paradigms, Dataset Composition, and Model Scaling for Child ASR in ESPnet [72.53502346791814]
  We compare flat-start training across datasets, SSL representations (WavLM, XEUS), and decoder architectures.<n> SSL representations are biased toward adult speech, with flat-start training on child speech mitigating these biases.<n>Age-related ASR and speaker verification analysis highlights the limitations of proprietary models.
  arXiv  Detail & Related papers  (2025-08-22T17:59:35Z)
• Layer-Wise Analysis of Self-Supervised Representations for Age and Gender Classification in Children's Speech [27.60599947546406]
  This paper presents a detailed analysis of four Wav2Vec2 variants using the PFSTAR and CMU Kids datasets.<n>Results show that early layers capture speaker-specific cues more effectively than deeper layers.<n>Applying PCA further improves classification, reducing redundancy and highlighting the most informative components.
  arXiv  Detail & Related papers  (2025-08-14T04:11:44Z)
• Evaluation of Speech Foundation Models for ASR on Child-Adult Conversations in Autism Diagnostic Sessions [27.30130353688078]
  Speech foundation models show a noticeable performance drop (15-20% absolute WER) for child speech compared to adult speech in the conversational setting.<n>We fine-tune the best-performing zero-shot model (Whisper-large) using LoRA in a low-resource setting, yielding 8% and 13% absolute WER improvements for child and adult speech, respectively.
  arXiv  Detail & Related papers  (2024-09-24T14:42:37Z)
• Homogeneous Speaker Features for On-the-Fly Dysarthric and Elderly Speaker Adaptation [71.31331402404662]
  This paper proposes two novel data-efficient methods to learn dysarthric and elderly speaker-level features. Speaker-regularized spectral basis embedding-SBE features that exploit a special regularization term to enforce homogeneity of speaker features in adaptation. Feature-based learning hidden unit contributions (f-LHUC) that are conditioned on VR-LH features that are shown to be insensitive to speaker-level data quantity in testtime adaptation.
  arXiv  Detail & Related papers  (2024-07-08T18:20:24Z)
• Self-supervised ASR Models and Features For Dysarthric and Elderly Speech Recognition [71.87998918300806]
  This paper explores approaches to integrate domain fine-tuned SSL pre-trained models and their features into TDNN and Conformer ASR systems. TDNN systems constructed by integrating domain-adapted HuBERT, wav2vec2-conformer or multi-lingual XLSR models consistently outperform standalone fine-tuned SSL pre-trained models. Consistent improvements in Alzheimer's Disease detection accuracy are also obtained using the DementiaBank Pitt elderly speech recognition outputs.
  arXiv  Detail & Related papers  (2024-07-03T08:33:39Z)
• A comparative analysis between Conformer-Transducer, Whisper, and wav2vec2 for improving the child speech recognition [2.965450563218781]
  We show that finetuning Conformer-transducer models on child speech yields significant improvements in ASR performance on child speech. We also show Whisper and wav2vec2 adaptation on different child speech datasets.
  arXiv  Detail & Related papers  (2023-11-07T19:32:48Z)
• Self-supervised models of audio effectively explain human cortical responses to speech [71.57870452667369]
  We capitalize on the progress of self-supervised speech representation learning to create new state-of-the-art models of the human auditory system. We show that these results show that self-supervised models effectively capture the hierarchy of information relevant to different stages of speech processing in human cortex.
  arXiv  Detail & Related papers  (2022-05-27T22:04:02Z)
• Self-Supervised Learning for speech recognition with Intermediate layer supervision [52.93758711230248]
  We propose Intermediate Layer Supervision for Self-Supervised Learning (ILS-SSL) ILS-SSL forces the model to concentrate on content information as much as possible by adding an additional SSL loss on the intermediate layers. Experiments on LibriSpeech test-other set show that our method outperforms HuBERT significantly.
  arXiv  Detail & Related papers  (2021-12-16T10:45:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.