Robust Emotion Recognition via Bi-Level Self-Supervised Continual Learning

• URL: http://arxiv.org/abs/2505.10575v2
• Date: Mon, 19 May 2025 02:10:28 GMT
• Title: Robust Emotion Recognition via Bi-Level Self-Supervised Continual Learning
• Abstract summary: Cross-subject variability and noisy labels hinder the performance of emotion recognition models.<n>We propose a novel bi-level self-supervised continual learning framework, SSOCL, based on a dynamic memory buffer.<n>This bi-level architecture iteratively refines the dynamic buffer and pseudo-label assignments to effectively retain representative samples.<n>Key components of the framework, including a fast adaptation module and a cluster-mapping module, enable robust learning and effective handling of evolving data streams.
• Score: 3.472622494096705
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Emotion recognition through physiological signals such as electroencephalogram (EEG) has become an essential aspect of affective computing and provides an objective way to capture human emotions. However, physiological data characterized by cross-subject variability and noisy labels hinder the performance of emotion recognition models. Existing domain adaptation and continual learning methods struggle to address these issues, especially under realistic conditions where data is continuously streamed and unlabeled. To overcome these limitations, we propose a novel bi-level self-supervised continual learning framework, SSOCL, based on a dynamic memory buffer. This bi-level architecture iteratively refines the dynamic buffer and pseudo-label assignments to effectively retain representative samples, enabling generalization from continuous, unlabeled physiological data streams for emotion recognition. The assigned pseudo-labels are subsequently leveraged for accurate emotion prediction. Key components of the framework, including a fast adaptation module and a cluster-mapping module, enable robust learning and effective handling of evolving data streams. Experimental validation on two mainstream EEG tasks demonstrates the framework's ability to adapt to continuous data streams while maintaining strong generalization across subjects, outperforming existing approaches.

Related papers

• Decoding Naturalistic Emotion Dynamics from the Brain: An LLM-Enhanced Regression Framework [3.8543064191088203]
  Decoding emotional states from neural signals has been typically framed as a discrete, single-label classification task.<n>This study reconceptualizes emotion decoding by adopting a multi-target regression framework to track multiple overlapping emotional dimensions as continuous trajectories over time.
  arXiv  Detail & Related papers  (2026-06-05T10:40:22Z)
• MA-DLE: Speech-based Automatic Depression Level Estimation via Memory Augmentation [59.709198999959206]
  Speech-based automatic estimation of depression levels is essential for enabling early detection and timely intervention.<n>Most existing approaches rely on RNN-based architectures to model temporal information for depression estimation.<n>We introduce a memory-based feature augmentation method that enhances the representational capacity of GRU-extracted features.
  arXiv  Detail & Related papers  (2026-04-20T14:42:13Z)
• Memory-guided Prototypical Co-occurrence Learning for Mixed Emotion Recognition [56.00118641432005]
  We propose a Memory-guided Prototypical Co-occurrence Learning framework that explicitly models emotion co-occurrence patterns.<n>Inspired by human cognitive memory systems, we introduce a memory retrieval strategy to extract semantic-level co-occurrence associations.<n>Our model learns affectively informative representations for accurate emotion distribution prediction.
  arXiv  Detail & Related papers  (2026-02-24T04:11:25Z)
• Adaptive Temporal Dynamics for Personalized Emotion Recognition: A Liquid Neural Network Approach [0.0]
  This work presents to the best of our knowledge, the first comprehensive application of liquid neural networks for EEG-based emotion recognition.<n>The proposed framework combines convolutional feature extraction, liquid neural networks with learnable time constants, and attention-guided fusion.<n>Subject-dependent experiments conducted on the PhyMER dataset across seven emotional classes achieve an accuracy of 95.45%, surpassing previously reported results.
  arXiv  Detail & Related papers  (2026-01-28T12:14:45Z)
• E^2-LLM: Bridging Neural Signals and Interpretable Affective Analysis [54.763420895859035]
  We present ELLM2-EEG-to-Emotion Large Language Model, first MLLM framework for interpretable emotion analysis from EEG.<n>ELLM integrates a pretrained EEG encoder with Q-based LLMs through learnable projection layers, employing a multi-stage training pipeline.<n>Experiments on the dataset across seven emotion categories demonstrate that ELLM2-EEG-to-Emotion Large Language Model achieves excellent performance on emotion classification.
  arXiv  Detail & Related papers  (2026-01-11T13:21:20Z)
• Cross-Modal Consistency-Guided Active Learning for Affective BCI Systems [1.9556470931534158]
  We propose an uncertainty-aware active learning framework that enhances robustness to label noise.<n>Instead of relying solely on EEG-based uncertainty estimates, the method evaluates cross-modal alignment.<n>This feedback-driven process guides the network toward reliable, informative samples and reduces the impact of noisy labels.
  arXiv  Detail & Related papers  (2025-11-19T05:33:48Z)
• Reinforced Interactive Continual Learning via Real-time Noisy Human Feedback [59.768119380109084]
  This paper introduces an interactive continual learning paradigm where AI models dynamically learn new skills from real-time human feedback.<n>We propose RiCL, a Reinforced interactive Continual Learning framework leveraging Large Language Models (LLMs)<n>Our RiCL approach substantially outperforms existing combinations of state-of-the-art online continual learning and noisy-label learning methods.
  arXiv  Detail & Related papers  (2025-05-15T03:22:03Z)
• Emotion Recognition with CLIP and Sequential Learning [5.66758879852618]
  We present our innovative methodology for tackling the Valence-Arousal (VA) Estimation Challenge, the Expression Recognition Challenge, and the Action Unit (AU) Detection Challenge.<n>Our approach introduces a novel framework aimed at enhancing continuous emotion recognition.
  arXiv  Detail & Related papers  (2025-03-13T01:02:06Z)
• Continuous Adversarial Text Representation Learning for Affective Recognition [1.319058156672392]
  We propose a novel framework for enhancing emotion-aware embeddings in transformer-based models.<n>Our approach introduces a continuous valence-arousal labeling system to guide contrastive learning.<n>We employ a dynamic token perturbation mechanism, using gradient-based saliency to focus on sentiment-relevant tokens, improving model sensitivity to emotional cues.
  arXiv  Detail & Related papers  (2025-02-28T00:29:09Z)
• Self-supervised Gait-based Emotion Representation Learning from Selective Strongly Augmented Skeleton Sequences [4.740624855896404]
  We propose a contrastive learning framework utilizing selective strong augmentation for self-supervised gait-based emotion representation. Our approach is validated on the Emotion-Gait (E-Gait) and Emilya datasets and outperforms the state-of-the-art methods under different evaluation protocols.
  arXiv  Detail & Related papers  (2024-05-08T09:13:10Z)
• Graph Convolutional Network with Connectivity Uncertainty for EEG-based Emotion Recognition [20.655367200006076]
  This study introduces the distribution-based uncertainty method to represent spatial dependencies and temporal-spectral relativeness in EEG signals. The graph mixup technique is employed to enhance latent connected edges and mitigate noisy label issues. We evaluate our approach on two widely used datasets, namely SEED and SEEDIV, for emotion recognition tasks.
  arXiv  Detail & Related papers  (2023-10-22T03:47:11Z)
• Transformer-based Self-supervised Multimodal Representation Learning for Wearable Emotion Recognition [2.4364387374267427]
  We propose a novel self-supervised learning (SSL) framework for wearable emotion recognition. Our method achieved state-of-the-art results in various emotion classification tasks.
  arXiv  Detail & Related papers  (2023-03-29T19:45:55Z)
• A Hierarchical Regression Chain Framework for Affective Vocal Burst Recognition [72.36055502078193]
  We propose a hierarchical framework, based on chain regression models, for affective recognition from vocal bursts. To address the challenge of data sparsity, we also use self-supervised learning (SSL) representations with layer-wise and temporal aggregation modules. The proposed systems participated in the ACII Affective Vocal Burst (A-VB) Challenge 2022 and ranked first in the "TWO'' and "CULTURE" tasks.
  arXiv  Detail & Related papers  (2023-03-14T16:08:45Z)
• Multimodal Emotion Recognition using Transfer Learning from Speaker Recognition and BERT-based models [53.31917090073727]
  We propose a neural network-based emotion recognition framework that uses a late fusion of transfer-learned and fine-tuned models from speech and text modalities. We evaluate the effectiveness of our proposed multimodal approach on the interactive emotional dyadic motion capture dataset.
  arXiv  Detail & Related papers  (2022-02-16T00:23:42Z)
• Improved Speech Emotion Recognition using Transfer Learning and Spectrogram Augmentation [56.264157127549446]
  Speech emotion recognition (SER) is a challenging task that plays a crucial role in natural human-computer interaction. One of the main challenges in SER is data scarcity. We propose a transfer learning strategy combined with spectrogram augmentation.
  arXiv  Detail & Related papers  (2021-08-05T10:39:39Z)
• Uncovering the structure of clinical EEG signals with self-supervised learning [64.4754948595556]
  Supervised learning paradigms are often limited by the amount of labeled data that is available. This phenomenon is particularly problematic in clinically-relevant data, such as electroencephalography (EEG) By extracting information from unlabeled data, it might be possible to reach competitive performance with deep neural networks.
  arXiv  Detail & Related papers  (2020-07-31T14:34:47Z)
• Continuous Emotion Recognition via Deep Convolutional Autoencoder and Support Vector Regressor [70.2226417364135]
  It is crucial that the machine should be able to recognize the emotional state of the user with high accuracy. Deep neural networks have been used with great success in recognizing emotions. We present a new model for continuous emotion recognition based on facial expression recognition.
  arXiv  Detail & Related papers  (2020-01-31T17:47:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.