Optimal Transport-based Adaptation in Dysarthric Speech Tasks

• URL: http://arxiv.org/abs/2104.02535v1
• Date: Tue, 6 Apr 2021 14:26:34 GMT
• Title: Optimal Transport-based Adaptation in Dysarthric Speech Tasks
• Abstract summary: In speech data, causes of mismatch include different acoustic environments or speaker characteristics. We propose an optimal-transport based approach, called MSDA via Weighted Joint Optimal Transport (MSDA-WDJOT) We employ MSDA-WJDOT for dysarthric speaker adaptation in command speech recognition.
• Score: 6.609497621455247
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In many real-world applications, the mismatch between distributions of training data (source) and test data (target) significantly degrades the performance of machine learning algorithms. In speech data, causes of this mismatch include different acoustic environments or speaker characteristics. In this paper, we address this issue in the challenging context of dysarthric speech, by multi-source domain/speaker adaptation (MSDA/MSSA). Specifically, we propose the use of an optimal-transport based approach, called MSDA via Weighted Joint Optimal Transport (MSDA-WDJOT). We confront the mismatch problem in dysarthria detection for which the proposed approach outperforms both the Baseline and the state-of-the-art MSDA models, improving the detection accuracy of 0.9% over the best competitor method. We then employ MSDA-WJDOT for dysarthric speaker adaptation in command speech recognition. This provides a Command Error Rate relative reduction of 16% and 7% over the baseline and the best competitor model, respectively. Interestingly, MSDA-WJDOT provides a similarity score between the source and the target, i.e. between speakers in this case. We leverage this similarity measure to define a Dysarthric and Healthy score of the target speaker and diagnose the dysarthria with an accuracy of 95%.

Related papers

• Improving End-to-End Speech Recognition for Dysarthric Speech through In-Domain Data Augmentation [43.31597557333867]
  Dysarthric speech recognition is crucial for facilitating effective communication among individuals with dysarthria.<n> accurately recognizing dysarthric speech poses significant challenges due to varying severity levels and limited data availability.<n>We explore data augmentation techniques for dysarthric automatic speech recognition (ASR) systems by fine-tuning the End-to-End pre-trained Wav2Vec2 model.
  arXiv  Detail & Related papers  (2026-06-18T05:00:11Z)
• Towards Personalized Federated Learning for Dysarthric Speech Recognition [49.291957451546914]
  This paper explores two aggregation strategies to achieve personalization, including the parameter-based averaging strategy and the embedding-based averaging strategy.<n> Experiments on UASpeech and TORGO show that the proposed methods outperform the baseline regularized FedAvg by statistically significant WER reductions of up to 0.99% absolute (3.15% relative) on UASpeech and 0.56% absolute (4.73% relative) on TORGO, respectively.
  arXiv  Detail & Related papers  (2026-06-11T12:10:57Z)
• Progressive Multi-Source Domain Adaptation for Personalized Facial Expression Recognition [51.61979855488214]
  Personalized facial expression recognition (FER) involves adapting a machine learning model using samples from labeled sources and unlabeled target domains. We propose a progressive MSDA approach that gradually introduces information from subjects based on their similarity to the target subject. Our experiments show the effectiveness of our proposed method on pain datasets: Biovid and UNBC-McMaster.
  arXiv  Detail & Related papers  (2025-04-05T19:14:51Z)
• Improved Diffusion-based Generative Model with Better Adversarial Robustness [65.38540020916432]
  Diffusion Probabilistic Models (DPMs) have achieved significant success in generative tasks. During the denoising process, the input data distributions differ between the training and inference stages.
  arXiv  Detail & Related papers  (2025-02-24T12:29:16Z)
• Towards Robust Alignment of Language Models: Distributionally Robustifying Direct Preference Optimization [45.6430987775264]
  This study addresses the challenge of noise in training datasets for Direct Preference Optimization (DPO) We categorize noise into pointwise noise, which includes low-quality data points, and pairwise noise, which encompasses erroneous data pair associations. We introduce Distributionally Robustifying DPO, which integrates pairwise robustness by optimizing against worst-case pairwise scenarios.
  arXiv  Detail & Related papers  (2024-07-10T17:48:25Z)
• A Study of Dropout-Induced Modality Bias on Robustness to Missing Video Frames for Audio-Visual Speech Recognition [53.800937914403654]
  Advanced Audio-Visual Speech Recognition (AVSR) systems have been observed to be sensitive to missing video frames. While applying the dropout technique to the video modality enhances robustness to missing frames, it simultaneously results in a performance loss when dealing with complete data input. We propose a novel Multimodal Distribution Approximation with Knowledge Distillation (MDA-KD) framework to reduce over-reliance on the audio modality.
  arXiv  Detail & Related papers  (2024-03-07T06:06:55Z)
• Improving Speaker-independent Speech Emotion Recognition Using Dynamic Joint Distribution Adaptation [15.64999039870125]
  In speaker-independent speech emotion recognition, the training and testing samples are collected from diverse speakers. When the trained model is confronted with data from new speakers, its performance tends to degrade. We propose a Dynamic Joint Distribution Adaptation (DJDA) method under the framework of multi-source domain adaptation.
  arXiv  Detail & Related papers  (2024-01-18T06:52:52Z)
• A Few-Shot Approach to Dysarthric Speech Intelligibility Level Classification Using Transformers [0.0]
  Dysarthria is a speech disorder that hinders communication due to difficulties in articulating words. Much of the literature focused on improving ASR systems for dysarthric speech. This work aims to develop models that can accurately classify the presence of dysarthria.
  arXiv  Detail & Related papers  (2023-09-17T17:23:41Z)
• Diffusion Denoising Process for Perceptron Bias in Out-of-distribution Detection [67.49587673594276]
  We introduce a new perceptron bias assumption that suggests discriminator models are more sensitive to certain features of the input, leading to the overconfidence problem. We demonstrate that the diffusion denoising process (DDP) of DMs serves as a novel form of asymmetric, which is well-suited to enhance the input and mitigate the overconfidence problem. Our experiments on CIFAR10, CIFAR100, and ImageNet show that our method outperforms SOTA approaches.
  arXiv  Detail & Related papers  (2022-11-21T08:45:08Z)
• A Comparative Study on Speaker-attributed Automatic Speech Recognition in Multi-party Meetings [53.120885867427305]
  Three approaches are evaluated for speaker-attributed automatic speech recognition (SA-ASR) in a meeting scenario. The WD-SOT approach achieves 10.7% relative reduction on averaged speaker-dependent character error rate (SD-CER) The TS-ASR approach also outperforms the FD-SOT approach and brings 16.5% relative average SD-CER reduction.
  arXiv  Detail & Related papers  (2022-03-31T06:39:14Z)
• On-the-Fly Feature Based Rapid Speaker Adaptation for Dysarthric and Elderly Speech Recognition [53.17176024917725]
  Scarcity of speaker-level data limits the practical use of data-intensive model based speaker adaptation methods. This paper proposes two novel forms of data-efficient, feature-based on-the-fly speaker adaptation methods.
  arXiv  Detail & Related papers  (2022-03-28T09:12:24Z)
• Interpretable Dysarthric Speaker Adaptation based on Optimal-Transport [3.903766260291157]
  We focus on Speaker Adaptation (SA) in command speech recognition, where data from multiple sources are available. We propose an unsupervised Multi-Source Domain Adaptation (MSDA) algorithm based on optimal-transport. We achieve a Command Error Rate relative reduction of 16% and 7% over the speaker-independent model and the best competitor method, respectively.
  arXiv  Detail & Related papers  (2022-03-14T14:39:00Z)
• Bures Joint Distribution Alignment with Dynamic Margin for Unsupervised Domain Adaptation [17.06364218327213]
  Unsupervised domain adaptation (UDA) is one of the prominent tasks of transfer learning. We propose a novel alignment loss term that minimizes the kernel Bures-Wasserstein distance between the joint distributions. Experiments show that BJDA is very effective for the UDA tasks, as it outperforms state-of-the-art algorithms in most experimental settings.
  arXiv  Detail & Related papers  (2022-03-14T03:20:01Z)
• UDALM: Unsupervised Domain Adaptation through Language Modeling [79.73916345178415]
  We introduce UDALM, a fine-tuning procedure, using a mixed classification and Masked Language Model loss. Our experiments show that performance of models trained with the mixed loss scales with the amount of available target data can be effectively used as a stopping criterion. Our method is evaluated on twelve domain pairs of the Amazon Reviews Sentiment dataset, yielding $91.74%$ accuracy, which is an $1.11%$ absolute improvement over the state-of-versathe-art.
  arXiv  Detail & Related papers  (2021-04-14T19:05:01Z)
• Towards a Competitive End-to-End Speech Recognition for CHiME-6 Dinner Party Transcription [73.66530509749305]
  In this paper, we argue that, even in difficult cases, some end-to-end approaches show performance close to the hybrid baseline. We experimentally compare and analyze CTC-Attention versus RNN-Transducer approaches along with RNN versus Transformer architectures. Our best end-to-end model based on RNN-Transducer, together with improved beam search, reaches quality by only 3.8% WER abs. worse than the LF-MMI TDNN-F CHiME-6 Challenge baseline.
  arXiv  Detail & Related papers  (2020-04-22T19:08:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.