Bayesian Learning for Deep Neural Network Adaptation

• URL: http://arxiv.org/abs/2012.07460v4
• Date: Fri, 14 Jun 2024 08:25:27 GMT
• Title: Bayesian Learning for Deep Neural Network Adaptation
• Authors: Xurong Xie, Xunying Liu, Tan Lee, Lan Wang,
• Abstract summary: A key task for speech recognition systems is to reduce the mismatch between training and evaluation data that is often attributable to speaker differences. Model-based speaker adaptation approaches often require sufficient amounts of target speaker data to ensure robustness. This paper proposes a full Bayesian learning based DNN speaker adaptation framework to model speaker-dependent (SD) parameter uncertainty.
• Score: 57.70991105736059
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: A key task for speech recognition systems is to reduce the mismatch between training and evaluation data that is often attributable to speaker differences. Speaker adaptation techniques play a vital role to reduce the mismatch. Model-based speaker adaptation approaches often require sufficient amounts of target speaker data to ensure robustness. When the amount of speaker level data is limited, speaker adaptation is prone to overfitting and poor generalization. To address the issue, this paper proposes a full Bayesian learning based DNN speaker adaptation framework to model speaker-dependent (SD) parameter uncertainty given limited speaker specific adaptation data. This framework is investigated in three forms of model based DNN adaptation techniques: Bayesian learning of hidden unit contributions (BLHUC), Bayesian parameterized activation functions (BPAct), and Bayesian hidden unit bias vectors (BHUB). In the three methods, deterministic SD parameters are replaced by latent variable posterior distributions for each speaker, whose parameters are efficiently estimated using a variational inference based approach. Experiments conducted on 300-hour speed perturbed Switchboard corpus trained LF-MMI TDNN/CNN-TDNN systems suggest the proposed Bayesian adaptation approaches consistently outperform the deterministic adaptation on the NIST Hub5'00 and RT03 evaluation sets. When using only the first five utterances from each speaker as adaptation data, significant word error rate reductions up to 1.4% absolute (7.2% relative) were obtained on the CallHome subset. The efficacy of the proposed Bayesian adaptation techniques is further demonstrated in a comparison against the state-of-the-art performance obtained on the same task using the most recent systems reported in the literature.

Related papers

• Homogeneous Speaker Features for On-the-Fly Dysarthric and Elderly Speaker Adaptation [71.31331402404662]
  This paper proposes two novel data-efficient methods to learn dysarthric and elderly speaker-level features. Speaker-regularized spectral basis embedding-SBE features that exploit a special regularization term to enforce homogeneity of speaker features in adaptation. Feature-based learning hidden unit contributions (f-LHUC) that are conditioned on VR-LH features that are shown to be insensitive to speaker-level data quantity in testtime adaptation.
  arXiv  Detail & Related papers  (2024-07-08T18:20:24Z)
• DASA: Difficulty-Aware Semantic Augmentation for Speaker Verification [55.306583814017046]
  We present a novel difficulty-aware semantic augmentation (DASA) approach for speaker verification. DASA generates diversified training samples in speaker embedding space with negligible extra computing cost. The best result achieves a 14.6% relative reduction in EER metric on CN-Celeb evaluation set.
  arXiv  Detail & Related papers  (2023-10-18T17:07:05Z)
• HyPoradise: An Open Baseline for Generative Speech Recognition with Large Language Models [81.56455625624041]
  We introduce the first open-source benchmark to utilize external large language models (LLMs) for ASR error correction. The proposed benchmark contains a novel dataset, HyPoradise (HP), encompassing more than 334,000 pairs of N-best hypotheses. LLMs with reasonable prompt and its generative capability can even correct those tokens that are missing in N-best list.
  arXiv  Detail & Related papers  (2023-09-27T14:44:10Z)
• Factorised Speaker-environment Adaptive Training of Conformer Speech Recognition Systems [31.813788489512394]
  This paper proposes a novel factorised speaker-environment adaptive training and test time adaptation approach for Conformer ASR models. Experiments on the 300-hr WHAM noise corrupted Switchboard data suggest that factorised adaptation consistently outperforms the baseline. Further analysis shows the proposed method offers potential for rapid adaption to unseen speaker-environment conditions.
  arXiv  Detail & Related papers  (2023-06-26T11:32:05Z)
• On-the-Fly Feature Based Rapid Speaker Adaptation for Dysarthric and Elderly Speech Recognition [53.17176024917725]
  Scarcity of speaker-level data limits the practical use of data-intensive model based speaker adaptation methods. This paper proposes two novel forms of data-efficient, feature-based on-the-fly speaker adaptation methods.
  arXiv  Detail & Related papers  (2022-03-28T09:12:24Z)
• Unsupervised Personalization of an Emotion Recognition System: The Unique Properties of the Externalization of Valence in Speech [37.6839508524855]
  Adapting a speech emotion recognition system to a particular speaker is a hard problem, especially with deep neural networks (DNNs) This study proposes an unsupervised approach to address this problem by searching for speakers in the train set with similar acoustic patterns as the speaker in the test set. We propose three alternative adaptation strategies: unique speaker, oversampling and weighting approaches.
  arXiv  Detail & Related papers  (2022-01-19T22:14:49Z)
• A Unified Speaker Adaptation Approach for ASR [37.76683818356052]
  We propose a unified speaker adaptation approach consisting of feature adaptation and model adaptation. For feature adaptation, we employ a speaker-aware persistent memory model which generalizes better to unseen test speakers. For model adaptation, we use a novel gradual pruning method to adapt to target speakers without changing the model architecture.
  arXiv  Detail & Related papers  (2021-10-16T10:48:52Z)
• Adapting Speaker Embeddings for Speaker Diarisation [30.383712356205084]
  The goal of this paper is to adapt speaker embeddings for solving the problem of speaker diarisation. We propose three techniques that can be used to better adapt the speaker embeddings for diarisation: dimensionality reduction, attention-based embedding aggregation, and non-speech clustering. The results demonstrate that all three techniques contribute positively to the performance of the diarisation system achieving an average relative improvement of 25.07% in terms of diarisation error rate over the baseline.
  arXiv  Detail & Related papers  (2021-04-07T03:04:47Z)
• Open-set Short Utterance Forensic Speaker Verification using Teacher-Student Network with Explicit Inductive Bias [59.788358876316295]
  We propose a pipeline solution to improve speaker verification on a small actual forensic field dataset. By leveraging large-scale out-of-domain datasets, a knowledge distillation based objective function is proposed for teacher-student learning. We show that the proposed objective function can efficiently improve the performance of teacher-student learning on short utterances.
  arXiv  Detail & Related papers  (2020-09-21T00:58:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.