Improving Speech Emotion Recognition with Mutual Information Regularized Generative Model

• URL: http://arxiv.org/abs/2510.10078v1
• Date: Sat, 11 Oct 2025 07:29:32 GMT
• Title: Improving Speech Emotion Recognition with Mutual Information Regularized Generative Model
• Abstract summary: We propose a data augmentation framework aided by cross-modal information transfer and mutual information regularization.<n>Our framework was tested on three benchmark datasets: IEMOCAP, MSP-IMPROV and MSP-Podcast.
• Score: 28.107097945179905
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Although speech emotion recognition (SER) research has been advanced, thanks to deep learning methods, it still suffers from obtaining inputs from large quality-labelled training data. Data augmentation methods have been attempted to mitigate this issue, generative models have shown success among them recently. We propose a data augmentation framework that is aided by cross-modal information transfer and mutual information regularization. Mutual information based metric can serve as an indicator for the quality. Furthermore, we expand this data augmentation scope to multimodal inputs, thanks to mutual information ensureing dependency between modalities. Our framework was tested on three benchmark datasets: IEMOCAP, MSP-IMPROV and MSP-Podcast. The implementation was designed to generate input features that are fed into last layer for emotion classification. Our framework improved the performance of emotion prediction against existing works. Also, we discovered that our framework is able to generate new inputs without any cross-modal information.

Related papers

• FedXDS: Leveraging Model Attribution Methods to counteract Data Heterogeneity in Federated Learning [16.981834481629885]
  FedXDS is the first approach to utilize feature attribution techniques to identify precisely which data elements should be selectively shared between clients.<n>We provide theoretical privacy guarantees and empirically demonstrate robustness against both membership inference and feature inversion attacks.
  arXiv  Detail & Related papers  (2026-06-30T14:35:45Z)
• ROSE: Retrieval-Oriented Segmentation Enhancement [79.7370065070812]
  Existing segmentation models based on multimodal large language models (MLLMs) often struggle with novel or emerging entities due to their inability to incorporate up-to-date knowledge.<n>We introduce the Novel Emerging Task (NEST), which focuses on segmenting (i) novel entities that MLLMs fail to recognize due to their absence from training data, and (ii) emerging entities that exist within the model's knowledge but demand up-to-date information for accurate recognition.<n>Rose: Retrieval-Oriented Enhancement, a plug-and-play framework designed to augment any MLLM-based segmentation model,
  arXiv  Detail & Related papers  (2026-04-15T17:59:35Z)
• Generative Data Augmentation in Graph Contrastive Learning for Recommendation [7.878752226948897]
  We propose a novel framework, GDA4Rec, to generate high-quality augmented views and provide robust self-supervised signals.<n>We employ a noise generation module that leverages deep generative models to approximate the distribution of original data for data augmentation.<n>GDA4Rec further extracts an item complement matrix to characterize the latent correlations between items and provide additional self-supervised signals.
  arXiv  Detail & Related papers  (2025-10-10T08:29:21Z)
• PropMEND: Hypernetworks for Knowledge Propagation in LLMs [82.99849359892112]
  We present a hypernetwork-based approach for knowledge propagation, named PropMEND.<n>We show almost 2x accuracy on challenging multi-hop questions whose answers are not explicitly stated in the injected fact.<n>We also introduce a new dataset, Controlled RippleEdit, to evaluate the generalization of our hypernetwork.
  arXiv  Detail & Related papers  (2025-06-10T15:44:19Z)
• SPADE: Structured Prompting Augmentation for Dialogue Enhancement in Machine-Generated Text Detection [15.626772502710867]
  We propose SPADE, a structured framework for detecting synthetic dialogues using prompt-based positive and negative samples.<n>Our proposed methods yield 14 new dialogue datasets, which we benchmark against eight MGT detection models.
  arXiv  Detail & Related papers  (2025-03-19T09:32:52Z)
• Web-Scale Visual Entity Recognition: An LLM-Driven Data Approach [56.55633052479446]
  Web-scale visual entity recognition presents significant challenges due to the lack of clean, large-scale training data. We propose a novel methodology to curate such a dataset, leveraging a multimodal large language model (LLM) for label verification, metadata generation, and rationale explanation. Experiments demonstrate that models trained on this automatically curated data achieve state-of-the-art performance on web-scale visual entity recognition tasks.
  arXiv  Detail & Related papers  (2024-10-31T06:55:24Z)
• Reinforcement Learning Based Multi-modal Feature Fusion Network for Novel Class Discovery [47.28191501836041]
  In this paper, we employ a Reinforcement Learning framework to simulate the cognitive processes of humans. We also deploy a Member-to-Leader Multi-Agent framework to extract and fuse features from multi-modal information. We demonstrate the performance of our approach in both the 3D and 2D domains by employing the OS-MN40, OS-MN40-Miss, and Cifar10 datasets.
  arXiv  Detail & Related papers  (2023-08-26T07:55:32Z)
• A Comparative Study of Data Augmentation Techniques for Deep Learning Based Emotion Recognition [11.928873764689458]
  We conduct a comprehensive evaluation of popular deep learning approaches for emotion recognition. We show that long-range dependencies in the speech signal are critical for emotion recognition. Speed/rate augmentation offers the most robust performance gain across models.
  arXiv  Detail & Related papers  (2022-11-09T17:27:03Z)
• Meta Knowledge Condensation for Federated Learning [65.20774786251683]
  Existing federated learning paradigms usually extensively exchange distributed models at a central solver to achieve a more powerful model. This would incur severe communication burden between a server and multiple clients especially when data distributions are heterogeneous. Unlike existing paradigms, we introduce an alternative perspective to significantly decrease the communication cost in federate learning.
  arXiv  Detail & Related papers  (2022-09-29T15:07:37Z)
• Attribute Inference Attack of Speech Emotion Recognition in Federated Learning Settings [56.93025161787725]
  Federated learning (FL) is a distributed machine learning paradigm that coordinates clients to train a model collaboratively without sharing local data. We propose an attribute inference attack framework that infers sensitive attribute information of the clients from shared gradients or model parameters. We show that the attribute inference attack is achievable for SER systems trained using FL.
  arXiv  Detail & Related papers  (2021-12-26T16:50:42Z)
• Recognizing More Emotions with Less Data Using Self-supervised Transfer Learning [0.0]
  We propose a novel transfer learning method for speech emotion recognition. With as low as 125 examples per emotion class, we were able to reach a higher accuracy than a strong baseline trained on 8 times more data.
  arXiv  Detail & Related papers  (2020-11-11T06:18:31Z)
• Omni-supervised Facial Expression Recognition via Distilled Data [120.11782405714234]
  We propose omni-supervised learning to exploit reliable samples in a large amount of unlabeled data for network training. We experimentally verify that the new dataset can significantly improve the ability of the learned FER model. To tackle this, we propose to apply a dataset distillation strategy to compress the created dataset into several informative class-wise images.
  arXiv  Detail & Related papers  (2020-05-18T09:36:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.