Improving Speaker-independent Speech Emotion Recognition Using Dynamic Joint Distribution Adaptation

• URL: http://arxiv.org/abs/2401.09752v1
• Date: Thu, 18 Jan 2024 06:52:52 GMT
• Title: Improving Speaker-independent Speech Emotion Recognition Using Dynamic Joint Distribution Adaptation
• Abstract summary: In speaker-independent speech emotion recognition, the training and testing samples are collected from diverse speakers. When the trained model is confronted with data from new speakers, its performance tends to degrade. We propose a Dynamic Joint Distribution Adaptation (DJDA) method under the framework of multi-source domain adaptation.
• Score: 15.64999039870125
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In speaker-independent speech emotion recognition, the training and testing samples are collected from diverse speakers, leading to a multi-domain shift challenge across the feature distributions of data from different speakers. Consequently, when the trained model is confronted with data from new speakers, its performance tends to degrade. To address the issue, we propose a Dynamic Joint Distribution Adaptation (DJDA) method under the framework of multi-source domain adaptation. DJDA firstly utilizes joint distribution adaptation (JDA), involving marginal distribution adaptation (MDA) and conditional distribution adaptation (CDA), to more precisely measure the multi-domain distribution shifts caused by different speakers. This helps eliminate speaker bias in emotion features, allowing for learning discriminative and speaker-invariant speech emotion features from coarse-level to fine-level. Furthermore, we quantify the adaptation contributions of MDA and CDA within JDA by using a dynamic balance factor based on $\mathcal{A}$-Distance, promoting to effectively handle the unknown distributions encountered in data from new speakers. Experimental results demonstrate the superior performance of our DJDA as compared to other state-of-the-art (SOTA) methods.

Related papers

• DAMSDAN: Distribution-Aware Multi-Source Domain Adaptation Network for Cross-Domain EEG-based Emotion Recognition [19.010493629153288]
  We propose a distribution-aware multi-source domain adaptation network (DAMSDAN) for emotion recognition.<n>DAMSDAN integrates prototype-based constraints with adversarial learning to drive the encoder toward discriminative, domain-invariant emotion representations.<n>Experiments on SEED and SEED-IV show average accuracies of 94.86% and 79.78% for cross-subject, and 95.12% and 83.15% for cross-session protocols.
  arXiv  Detail & Related papers  (2025-10-20T12:18:46Z)
• High-Quality Sound Separation Across Diverse Categories via Visually-Guided Generative Modeling [65.02357548201188]
  We propose DAVIS, a Diffusion-based Audio-VIsual Separation framework that solves the audio-visual sound source separation task through generative learning.<n>Our framework operates by synthesizing the desired separated sound spectrograms directly from a noise distribution, conditioned concurrently on the mixed audio input and associated visual information.
  arXiv  Detail & Related papers  (2025-09-26T08:46:00Z)
• CO-VADA: A Confidence-Oriented Voice Augmentation Debiasing Approach for Fair Speech Emotion Recognition [49.27067541740956]
  We present CO-VADA, a Confidence-Oriented Voice Augmentation Debiasing Approach that mitigates bias without modifying model architecture or relying on demographic information.<n>CO-VADA identifies training samples that reflect bias patterns present in the training data and then applies voice conversion to alter irrelevant attributes and generate samples.<n>Our framework is compatible with various SER models and voice conversion tools, making it a scalable and practical solution for improving fairness in SER systems.
  arXiv  Detail & Related papers  (2025-06-06T13:25:56Z)
• Efficient Distribution Matching of Representations via Noise-Injected Deep InfoMax [73.03684002513218]
  We enhance Deep InfoMax (DIM) to enable automatic matching of learned representations to a selected prior distribution. We show that such modification allows for learning uniformly and normally distributed representations. The results indicate a moderate trade-off between the performance on the downstream tasks and quality of DM.
  arXiv  Detail & Related papers  (2024-10-09T15:40:04Z)
• Adversarial Domain Adaptation for Cross-user Activity Recognition Using Diffusion-based Noise-centred Learning [0.0]
  Human Activity Recognition (HAR) plays a crucial role in various applications such as human-computer interaction and healthcare monitoring. This paper introduces a novel framework, termed Diffusion-based Noise-centered Adrial Learning Domain Adaptation (Diff-Noise-Adv-DA) It is designed to address these challenges by leveraging generative diffusion modeling and adversarial learning techniques.
  arXiv  Detail & Related papers  (2024-08-06T08:55:49Z)
• DASA: Difficulty-Aware Semantic Augmentation for Speaker Verification [55.306583814017046]
  We present a novel difficulty-aware semantic augmentation (DASA) approach for speaker verification. DASA generates diversified training samples in speaker embedding space with negligible extra computing cost. The best result achieves a 14.6% relative reduction in EER metric on CN-Celeb evaluation set.
  arXiv  Detail & Related papers  (2023-10-18T17:07:05Z)
• A Comparative Study on Speaker-attributed Automatic Speech Recognition in Multi-party Meetings [53.120885867427305]
  Three approaches are evaluated for speaker-attributed automatic speech recognition (SA-ASR) in a meeting scenario. The WD-SOT approach achieves 10.7% relative reduction on averaged speaker-dependent character error rate (SD-CER) The TS-ASR approach also outperforms the FD-SOT approach and brings 16.5% relative average SD-CER reduction.
  arXiv  Detail & Related papers  (2022-03-31T06:39:14Z)
• On-the-Fly Feature Based Rapid Speaker Adaptation for Dysarthric and Elderly Speech Recognition [53.17176024917725]
  Scarcity of speaker-level data limits the practical use of data-intensive model based speaker adaptation methods. This paper proposes two novel forms of data-efficient, feature-based on-the-fly speaker adaptation methods.
  arXiv  Detail & Related papers  (2022-03-28T09:12:24Z)
• Interpretable Dysarthric Speaker Adaptation based on Optimal-Transport [3.903766260291157]
  We focus on Speaker Adaptation (SA) in command speech recognition, where data from multiple sources are available. We propose an unsupervised Multi-Source Domain Adaptation (MSDA) algorithm based on optimal-transport. We achieve a Command Error Rate relative reduction of 16% and 7% over the speaker-independent model and the best competitor method, respectively.
  arXiv  Detail & Related papers  (2022-03-14T14:39:00Z)
• Bures Joint Distribution Alignment with Dynamic Margin for Unsupervised Domain Adaptation [17.06364218327213]
  Unsupervised domain adaptation (UDA) is one of the prominent tasks of transfer learning. We propose a novel alignment loss term that minimizes the kernel Bures-Wasserstein distance between the joint distributions. Experiments show that BJDA is very effective for the UDA tasks, as it outperforms state-of-the-art algorithms in most experimental settings.
  arXiv  Detail & Related papers  (2022-03-14T03:20:01Z)
• Optimal Transport-based Adaptation in Dysarthric Speech Tasks [6.609497621455247]
  In speech data, causes of mismatch include different acoustic environments or speaker characteristics. We propose an optimal-transport based approach, called MSDA via Weighted Joint Optimal Transport (MSDA-WDJOT) We employ MSDA-WJDOT for dysarthric speaker adaptation in command speech recognition.
  arXiv  Detail & Related papers  (2021-04-06T14:26:34Z)
• Bayesian Learning for Deep Neural Network Adaptation [57.70991105736059]
  A key task for speech recognition systems is to reduce the mismatch between training and evaluation data that is often attributable to speaker differences. Model-based speaker adaptation approaches often require sufficient amounts of target speaker data to ensure robustness. This paper proposes a full Bayesian learning based DNN speaker adaptation framework to model speaker-dependent (SD) parameter uncertainty.
  arXiv  Detail & Related papers  (2020-12-14T12:30:41Z)
• DEAAN: Disentangled Embedding and Adversarial Adaptation Network for Robust Speaker Representation Learning [69.70594547377283]
  We propose a novel framework to disentangle speaker-related and domain-specific features. Our framework can effectively generate more speaker-discriminative and domain-invariant speaker representations.
  arXiv  Detail & Related papers  (2020-12-12T19:46:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.