Learning from Annotation Uncertainty: Entropy-Aware Curriculum for Speech Emotion Recognition

• URL: http://arxiv.org/abs/2606.27536v1
• Date: Thu, 25 Jun 2026 20:35:17 GMT
• Title: Learning from Annotation Uncertainty: Entropy-Aware Curriculum for Speech Emotion Recognition
• Abstract summary: Speech emotion recognition (SER) often relies on hard consensus labels that collapse annotator disagreement.<n>We study distribution-based supervision for 9-class SER on MSP-Podcast 2.0 using a WavLM-Base multitask model for categorical emotion and dimensional VAD.
• Score: 28.280450577162622
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Speech emotion recognition (SER) often relies on hard consensus labels that collapse annotator disagreement. We study distribution-based supervision for 9-class SER on MSP-Podcast 2.0 using a WavLM-Base multitask model for categorical emotion and dimensional VAD. Hard-label training is compared with targets from primary and merged primary--secondary annotator vote distributions. Distributional objectives improve alignment with human vote distributions, reducing JSD/KLD relative to hard-label training. Analysis shows that hard supervision partly benefits from assigning ambiguous utterances to the residual Other class, whereas distributional supervision redistributes uncertainty across emotion categories. Entropy-stratified evaluation shows that high-ambiguity utterances remain challenging, but distribution-based supervision better captures perceptual uncertainty. These findings support moving beyond hard labels toward targets that reflect listener disagreement.

Related papers

• AffectOmni: RL-Verifiable People-Centric Grounded Affective Reasoning for Social and Art-Related Scenes [59.134503227559634]
  Multimodal large language models (MLLMs) achieve strong performance on VQA and scene understanding.<n>Models may predict correct answers while neglecting people-centric cues such as micro expressions and body language.<n>We propose Affect Omni, a framework for verifiable affective reasoning.
  arXiv  Detail & Related papers  (2026-08-24T11:25:04Z)
• Reasoning under Ambiguity: Uncertainty-Aware Multilingual Emotion Classification under Partial Supervision [0.4925222726301579]
  This paper introduces Reasoning under Ambiguity, an uncertainty-aware framework for multilingual multi-label emotion classification.<n> Experiments on English, Spanish, and Arabic emotion classification benchmarks demonstrate consistent improvements over strong baselines.
  arXiv  Detail & Related papers  (2026-02-05T09:17:25Z)
• Soft-Label Training Preserves Epistemic Uncertainty [4.2541480167286005]
  Machine learning tasks involve inherent subjectivity, where annotators naturally provide varied labels.<n>Standard practice collapses these label distributions into single labels, aggregating diverse human judgments into point estimates.<n>We argue that this approach is misaligned for ambiguous data--the annotation distribution itself should be regarded as the ground truth.
  arXiv  Detail & Related papers  (2025-11-18T04:02:29Z)
• Mitigating Subgroup Disparities in Multi-Label Speech Emotion Recognition: A Pseudo-Labeling and Unsupervised Learning Approach [53.824673312331626]
  Implicit Demography Inference (IDI) module uses k-means clustering to mitigate bias in Speech Emotion Recognition (SER)<n>Experiments show that pseudo-labeling IDI reduces subgroup disparities, improving fairness metrics by over 28%.<n>Unsupervised IDI yields more than a 4.6% improvement in fairness metrics with a drop of less than 3.6% in SER performance.
  arXiv  Detail & Related papers  (2025-05-20T14:50:44Z)
• Handling Ambiguity in Emotion: From Out-of-Domain Detection to Distribution Estimation [45.53789836426869]
  The subjective perception of emotion leads to inconsistent labels from human annotators. This paper investigates three methods to handle ambiguous emotion. We show that incorporating utterances without majority-agreed labels as an additional class in the classifier reduces the classification performance of the other emotion classes. We also propose detecting utterances with ambiguous emotions as out-of-domain samples by quantifying the uncertainty in emotion classification using evidential deep learning.
  arXiv  Detail & Related papers  (2024-02-20T09:53:38Z)
• Robust Representation Learning for Unreliable Partial Label Learning [86.909511808373]
  Partial Label Learning (PLL) is a type of weakly supervised learning where each training instance is assigned a set of candidate labels, but only one label is the ground-truth. This is known as Unreliable Partial Label Learning (UPLL) that introduces an additional complexity due to the inherent unreliability and ambiguity of partial labels. We propose the Unreliability-Robust Representation Learning framework (URRL) that leverages unreliability-robust contrastive learning to help the model fortify against unreliable partial labels effectively.
  arXiv  Detail & Related papers  (2023-08-31T13:37:28Z)
• Unilaterally Aggregated Contrastive Learning with Hierarchical Augmentation for Anomaly Detection [64.50126371767476]
  We propose Unilaterally Aggregated Contrastive Learning with Hierarchical Augmentation (UniCon-HA) We explicitly encourage the concentration of inliers and the dispersion of virtual outliers via supervised and unsupervised contrastive losses. Our method is evaluated under three AD settings including unlabeled one-class, unlabeled multi-class, and labeled multi-class.
  arXiv  Detail & Related papers  (2023-08-20T04:01:50Z)
• Ambiguity-Resistant Semi-Supervised Learning for Dense Object Detection [98.66771688028426]
  We propose a Ambiguity-Resistant Semi-supervised Learning (ARSL) for one-stage detectors. Joint-Confidence Estimation (JCE) is proposed to quantifies the classification and localization quality of pseudo labels. ARSL effectively mitigates the ambiguities and achieves state-of-the-art SSOD performance on MS COCO and PASCAL VOC.
  arXiv  Detail & Related papers  (2023-03-27T07:46:58Z)
• End-to-End Label Uncertainty Modeling in Speech Emotion Recognition using Bayesian Neural Networks and Label Distribution Learning [0.0]
  We propose an end-to-end Bayesian neural network capable of being trained on a distribution of annotations to capture the subjectivity-based label uncertainty. We show that the proposed t-distribution based approach achieves state-of-the-art uncertainty modeling results in speech emotion recognition.
  arXiv  Detail & Related papers  (2022-09-30T12:55:43Z)
• Seeking Subjectivity in Visual Emotion Distribution Learning [93.96205258496697]
  Visual Emotion Analysis (VEA) aims to predict people's emotions towards different visual stimuli. Existing methods often predict visual emotion distribution in a unified network, neglecting the inherent subjectivity in its crowd voting process. We propose a novel textitSubjectivity Appraise-and-Match Network (SAMNet) to investigate the subjectivity in visual emotion distribution.
  arXiv  Detail & Related papers  (2022-07-25T02:20:03Z)
• Estimating the Uncertainty in Emotion Class Labels with Utterance-Specific Dirichlet Priors [24.365876333182207]
  We propose a novel training loss based on per-utterance Dirichlet prior distributions for verbal emotion recognition. An additional metric is used to evaluate the performance by detecting test utterances with high labelling uncertainty. Experiments with the widely used IEMOCAP dataset demonstrate that the two-branch structure achieves state-of-the-art classification results.
  arXiv  Detail & Related papers  (2022-03-08T23:30:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.