Test-Time Adaptation for Speech Emotion Recognition

• URL: http://arxiv.org/abs/2601.16240v1
• Date: Wed, 21 Jan 2026 23:45:17 GMT
• Title: Test-Time Adaptation for Speech Emotion Recognition
• Authors: Jiaheng Dong, Hong Jia, Ting Dang,
• Abstract summary: Test-time adaptation (TTA) bridges the gap by adapting models at inference using only unlabeled target data.<n>We present the first systematic evaluation and comparison covering 11 TTA methods across three representative SER tasks.<n>The results indicate that backpropagation-free TTA methods are the most promising.<n>No single method universally excels, and its effectiveness is highly dependent on the distributional shifts and tasks.
• Score: 8.729816363600348
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The practical utility of Speech Emotion Recognition (SER) systems is undermined by their fragility to domain shifts, such as speaker variability, the distinction between acted and naturalistic emotions, and cross-corpus variations. While domain adaptation and fine-tuning are widely studied, they require either source data or labelled target data, which are often unavailable or raise privacy concerns in SER. Test-time adaptation (TTA) bridges this gap by adapting models at inference using only unlabeled target data. Yet, having been predominantly designed for image classification and speech recognition, the efficacy of TTA for mitigating the unique domain shifts in SER has not been investigated. In this paper, we present the first systematic evaluation and comparison covering 11 TTA methods across three representative SER tasks. The results indicate that backpropagation-free TTA methods are the most promising. Conversely, entropy minimization and pseudo-labeling generally fail, as their core assumption of a single, confident ground-truth label is incompatible with the inherent ambiguity of emotional expression. Further, no single method universally excels, and its effectiveness is highly dependent on the distributional shifts and tasks.

Related papers

• EMO-Debias: Benchmarking Gender Debiasing Techniques in Multi-Label Speech Emotion Recognition [49.27067541740956]
  EMO-Debias is a large-scale comparison of 13 debiasing methods applied to multi-label SER.<n>Our study encompasses techniques from pre-processing, regularization, adversarial learning, biased learners, and distributionally robust optimization.<n>Our analysis quantifies the trade-offs between fairness and accuracy, identifying which approaches consistently reduce gender performance gaps.
  arXiv  Detail & Related papers  (2025-06-05T05:48:31Z)
• Towards Practical Emotion Recognition: An Unsupervised Source-Free Approach for EEG Domain Adaptation [0.5755004576310334]
  We propose a novel SF-UDA approach for EEG-based emotion classification across domains.<n>We introduce Dual-Loss Adaptive Regularization (DLAR) to minimize prediction discrepancies and align predictions with expected pseudo-labels.<n>Our approach significantly outperforms state-of-the-art methods, achieving 65.84% accuracy when trained on DEAP and tested on SEED.
  arXiv  Detail & Related papers  (2025-03-26T14:29:20Z)
• Fuzzy-aware Loss for Source-free Domain Adaptation in Visual Emotion Recognition [17.617780698468465]
  Source-free domain adaptation in visual emotion recognition (SFDA-VER) is a highly challenging task.<n>We propose a novel fuzzy-aware loss (FAL) to enable the VER model to better learn and adapt to new domains under fuzzy labels.
  arXiv  Detail & Related papers  (2025-01-26T13:20:52Z)
• Self-supervised Gait-based Emotion Representation Learning from Selective Strongly Augmented Skeleton Sequences [4.740624855896404]
  We propose a contrastive learning framework utilizing selective strong augmentation for self-supervised gait-based emotion representation. Our approach is validated on the Emotion-Gait (E-Gait) and Emilya datasets and outperforms the state-of-the-art methods under different evaluation protocols.
  arXiv  Detail & Related papers  (2024-05-08T09:13:10Z)
• MSAC: Multiple Speech Attribute Control Method for Reliable Speech Emotion Recognition [7.81011775615268]
  We introduce MSAC-SERNet, a novel unified SER framework capable of simultaneously handling both single-corpus and cross-corpus SER. Considering information overlap between various speech attributes, we propose a novel learning paradigm based on correlations of different speech attributes. Experiments on both single-corpus and cross-corpus SER scenarios indicate that MSAC-SERNet achieves superior performance compared to state-of-the-art SER approaches.
  arXiv  Detail & Related papers  (2023-08-08T03:43:24Z)
• Cluster-level pseudo-labelling for source-free cross-domain facial expression recognition [94.56304526014875]
  We propose the first Source-Free Unsupervised Domain Adaptation (SFUDA) method for Facial Expression Recognition (FER) Our method exploits self-supervised pretraining to learn good feature representations from the target data. We validate the effectiveness of our method in four adaptation setups, proving that it consistently outperforms existing SFUDA methods when applied to FER.
  arXiv  Detail & Related papers  (2022-10-11T08:24:50Z)
• Multimodal Emotion Recognition with Modality-Pairwise Unsupervised Contrastive Loss [80.79641247882012]
  We focus on unsupervised feature learning for Multimodal Emotion Recognition (MER) We consider discrete emotions, and as modalities text, audio and vision are used. Our method, as being based on contrastive loss between pairwise modalities, is the first attempt in MER literature.
  arXiv  Detail & Related papers  (2022-07-23T10:11:24Z)
• Balancing Discriminability and Transferability for Source-Free Domain Adaptation [55.143687986324935]
  Conventional domain adaptation (DA) techniques aim to improve domain transferability by learning domain-invariant representations. The requirement of simultaneous access to labeled source and unlabeled target renders them unsuitable for the challenging source-free DA setting. We derive novel insights to show that a mixup between original and corresponding translated generic samples enhances the discriminability-transferability trade-off.
  arXiv  Detail & Related papers  (2022-06-16T09:06:22Z)
• Listen, Adapt, Better WER: Source-free Single-utterance Test-time Adaptation for Automatic Speech Recognition [65.84978547406753]
  Test-time Adaptation aims to adapt the model trained on source domains to yield better predictions for test samples. Single-Utterance Test-time Adaptation (SUTA) is the first TTA study in speech area to our best knowledge.
  arXiv  Detail & Related papers  (2022-03-27T06:38:39Z)
• Estimating the Uncertainty in Emotion Class Labels with Utterance-Specific Dirichlet Priors [24.365876333182207]
  We propose a novel training loss based on per-utterance Dirichlet prior distributions for verbal emotion recognition. An additional metric is used to evaluate the performance by detecting test utterances with high labelling uncertainty. Experiments with the widely used IEMOCAP dataset demonstrate that the two-branch structure achieves state-of-the-art classification results.
  arXiv  Detail & Related papers  (2022-03-08T23:30:01Z)
• Unsupervised Domain Adaptation for Speech Recognition via Uncertainty Driven Self-Training [55.824641135682725]
  Domain adaptation experiments using WSJ as a source domain and TED-LIUM 3 as well as SWITCHBOARD show that up to 80% of the performance of a system trained on ground-truth data can be recovered.
  arXiv  Detail & Related papers  (2020-11-26T18:51:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.