Interpretable Dysarthric Speaker Adaptation based on Optimal-Transport

• URL: http://arxiv.org/abs/2203.07143v1
• Date: Mon, 14 Mar 2022 14:39:00 GMT
• Title: Interpretable Dysarthric Speaker Adaptation based on Optimal-Transport
• Abstract summary: We focus on Speaker Adaptation (SA) in command speech recognition, where data from multiple sources are available. We propose an unsupervised Multi-Source Domain Adaptation (MSDA) algorithm based on optimal-transport. We achieve a Command Error Rate relative reduction of 16% and 7% over the speaker-independent model and the best competitor method, respectively.
• Score: 3.903766260291157
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This work addresses the mismatch problem between the distribution of training data (source) and testing data (target), in the challenging context of dysarthric speech recognition. We focus on Speaker Adaptation (SA) in command speech recognition, where data from multiple sources (i.e., multiple speakers) are available. Specifically, we propose an unsupervised Multi-Source Domain Adaptation (MSDA) algorithm based on optimal-transport, called MSDA via Weighted Joint Optimal Transport (MSDA-WJDOT). We achieve a Command Error Rate relative reduction of 16% and 7% over the speaker-independent model and the best competitor method, respectively. The strength of the proposed approach is that, differently from any other existing SA method, it offers an interpretable model that can also be exploited, in this context, to diagnose dysarthria without any specific training. Indeed, it provides a closeness measure between the target and the source speakers, reflecting their similarity in terms of speech characteristics. Based on the similarity between the target speaker and the healthy/dysarthric source speakers, we then define the healthy/dysarthric score of the target speaker that we leverage to perform dysarthria detection. This approach does not require any additional training and achieves a 95% accuracy in the dysarthria diagnosis.

Related papers

• Towards Personalized Federated Learning for Dysarthric Speech Recognition [49.291957451546914]
  This paper explores two aggregation strategies to achieve personalization, including the parameter-based averaging strategy and the embedding-based averaging strategy.<n> Experiments on UASpeech and TORGO show that the proposed methods outperform the baseline regularized FedAvg by statistically significant WER reductions of up to 0.99% absolute (3.15% relative) on UASpeech and 0.56% absolute (4.73% relative) on TORGO, respectively.
  arXiv  Detail & Related papers  (2026-06-11T12:10:57Z)
• $C^2$AV-TSE: Context and Confidence-aware Audio Visual Target Speaker Extraction [80.57232374640911]
  We propose a model-agnostic strategy called the Mask-And-Recover (MAR) MAR integrates both inter- and intra-modality contextual correlations to enable global inference within extraction modules. To better target challenging parts within each sample, we introduce a Fine-grained Confidence Score (FCS) model.
  arXiv  Detail & Related papers  (2025-04-01T13:01:30Z)
• Robust Cross-Etiology and Speaker-Independent Dysarthric Speech Recognition [26.26414139359157]
  We present a speaker-independent dysarthric speech recognition system, with a focus on evaluating the recently released Speech Accessibility Project (SAP-1005) dataset. Our primary objective is to develop a robust speaker-independent model capable of accurately recognizing dysarthric speech, irrespective of the speaker. As a secondary objective, we aim to test the cross-etiology performance of our model by evaluating it on the TORGO dataset.
  arXiv  Detail & Related papers  (2025-01-25T00:02:58Z)
• A Multi-modal Approach to Dysarthria Detection and Severity Assessment Using Speech and Text Information [9.172160338245252]
  This study introduces a novel approach that leverages both speech and text modalities. By employing cross-attention mechanism, our method learns the acoustic and linguistic similarities between speech and text representations.
  arXiv  Detail & Related papers  (2024-12-22T06:08:35Z)
• Improving Speaker-independent Speech Emotion Recognition Using Dynamic Joint Distribution Adaptation [15.64999039870125]
  In speaker-independent speech emotion recognition, the training and testing samples are collected from diverse speakers. When the trained model is confronted with data from new speakers, its performance tends to degrade. We propose a Dynamic Joint Distribution Adaptation (DJDA) method under the framework of multi-source domain adaptation.
  arXiv  Detail & Related papers  (2024-01-18T06:52:52Z)
• Detecting Speech Abnormalities with a Perceiver-based Sequence Classifier that Leverages a Universal Speech Model [4.503292461488901]
  We propose a Perceiver-based sequence to detect abnormalities in speech reflective of several neurological disorders. We combine this sequence with a Universal Speech Model (USM) that is trained (unsupervised) on 12 million hours of diverse audio recordings. Our model outperforms standard transformer (80.9%) and perceiver (81.8%) models and achieves an average accuracy of 83.1%.
  arXiv  Detail & Related papers  (2023-10-16T21:07:12Z)
• A Few-Shot Approach to Dysarthric Speech Intelligibility Level Classification Using Transformers [0.0]
  Dysarthria is a speech disorder that hinders communication due to difficulties in articulating words. Much of the literature focused on improving ASR systems for dysarthric speech. This work aims to develop models that can accurately classify the presence of dysarthria.
  arXiv  Detail & Related papers  (2023-09-17T17:23:41Z)
• Context-aware attention layers coupled with optimal transport domain adaptation and multimodal fusion methods for recognizing dementia from spontaneous speech [0.0]
  Alzheimer's disease (AD) constitutes a complex neurocognitive disease and is the main cause of dementia. We propose some new methods for detecting AD patients, which capture the intra- and cross-modal interactions. Experiments conducted on the ADReSS and ADReSSo Challenge indicate the efficacy of our introduced approaches over existing research initiatives.
  arXiv  Detail & Related papers  (2023-05-25T18:18:09Z)
• A Comparative Study on Speaker-attributed Automatic Speech Recognition in Multi-party Meetings [53.120885867427305]
  Three approaches are evaluated for speaker-attributed automatic speech recognition (SA-ASR) in a meeting scenario. The WD-SOT approach achieves 10.7% relative reduction on averaged speaker-dependent character error rate (SD-CER) The TS-ASR approach also outperforms the FD-SOT approach and brings 16.5% relative average SD-CER reduction.
  arXiv  Detail & Related papers  (2022-03-31T06:39:14Z)
• On-the-Fly Feature Based Rapid Speaker Adaptation for Dysarthric and Elderly Speech Recognition [53.17176024917725]
  Scarcity of speaker-level data limits the practical use of data-intensive model based speaker adaptation methods. This paper proposes two novel forms of data-efficient, feature-based on-the-fly speaker adaptation methods.
  arXiv  Detail & Related papers  (2022-03-28T09:12:24Z)
• Speaker Identity Preservation in Dysarthric Speech Reconstruction by Adversarial Speaker Adaptation [59.41186714127256]
  Dysarthric speech reconstruction (DSR) aims to improve the quality of dysarthric speech. Speaker encoder (SE) optimized for speaker verification has been explored to control the speaker identity. We propose a novel multi-task learning strategy, i.e., adversarial speaker adaptation (ASA)
  arXiv  Detail & Related papers  (2022-02-18T08:59:36Z)
• Investigation of Data Augmentation Techniques for Disordered Speech Recognition [69.50670302435174]
  This paper investigates a set of data augmentation techniques for disordered speech recognition. Both normal and disordered speech were exploited in the augmentation process. The final speaker adapted system constructed using the UASpeech corpus and the best augmentation approach based on speed perturbation produced up to 2.92% absolute word error rate (WER)
  arXiv  Detail & Related papers  (2022-01-14T17:09:22Z)
• Optimal Transport-based Adaptation in Dysarthric Speech Tasks [6.609497621455247]
  In speech data, causes of mismatch include different acoustic environments or speaker characteristics. We propose an optimal-transport based approach, called MSDA via Weighted Joint Optimal Transport (MSDA-WDJOT) We employ MSDA-WJDOT for dysarthric speaker adaptation in command speech recognition.
  arXiv  Detail & Related papers  (2021-04-06T14:26:34Z)
• NUVA: A Naming Utterance Verifier for Aphasia Treatment [49.114436579008476]
  Assessment of speech performance using picture naming tasks is a key method for both diagnosis and monitoring of responses to treatment interventions by people with aphasia (PWA) Here we present NUVA, an utterance verification system incorporating a deep learning element that classifies 'correct' versus'incorrect' naming attempts from aphasic stroke patients. When tested on eight native British-English speaking PWA the system's performance accuracy ranged between 83.6% to 93.6%, with a 10-fold cross-validation mean of 89.5%.
  arXiv  Detail & Related papers  (2021-02-10T13:00:29Z)
• Bayesian Learning for Deep Neural Network Adaptation [57.70991105736059]
  A key task for speech recognition systems is to reduce the mismatch between training and evaluation data that is often attributable to speaker differences. Model-based speaker adaptation approaches often require sufficient amounts of target speaker data to ensure robustness. This paper proposes a full Bayesian learning based DNN speaker adaptation framework to model speaker-dependent (SD) parameter uncertainty.
  arXiv  Detail & Related papers  (2020-12-14T12:30:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.