Complex Emotion Recognition System using basic emotions via Facial Expression, EEG, and ECG Signals: a review

• URL: http://arxiv.org/abs/2409.07493v1
• Date: Mon, 9 Sep 2024 05:06:10 GMT
• Title: Complex Emotion Recognition System using basic emotions via Facial Expression, EEG, and ECG Signals: a review
• Authors: Javad Hassannataj Joloudari, Mohammad Maftoun, Bahareh Nakisa, Roohallah Alizadehsani, Meisam Yadollahzadeh-Tabari,
• Abstract summary: The Complex Emotion Recognition System (CERS) deciphers complex emotional states by examining combinations of basic emotions expressed, their interconnections, and the dynamic variations. The development of AI systems for discerning complex emotions poses a substantial challenge with significant implications for affective computing. incorporating physiological signals such as Electrocardiogram (ECG) and Electroencephalogram (EEG) can notably enhance CERS.
• Score: 1.8310098790941458
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: The Complex Emotion Recognition System (CERS) deciphers complex emotional states by examining combinations of basic emotions expressed, their interconnections, and the dynamic variations. Through the utilization of advanced algorithms, CERS provides profound insights into emotional dynamics, facilitating a nuanced understanding and customized responses. The attainment of such a level of emotional recognition in machines necessitates the knowledge distillation and the comprehension of novel concepts akin to human cognition. The development of AI systems for discerning complex emotions poses a substantial challenge with significant implications for affective computing. Furthermore, obtaining a sizable dataset for CERS proves to be a daunting task due to the intricacies involved in capturing subtle emotions, necessitating specialized methods for data collection and processing. Incorporating physiological signals such as Electrocardiogram (ECG) and Electroencephalogram (EEG) can notably enhance CERS by furnishing valuable insights into the user's emotional state, enhancing the quality of datasets, and fortifying system dependability. A comprehensive literature review was conducted in this study to assess the efficacy of machine learning, deep learning, and meta-learning approaches in both basic and complex emotion recognition utilizing EEG, ECG signals, and facial expression datasets. The chosen research papers offer perspectives on potential applications, clinical implications, and results of CERSs, with the objective of promoting their acceptance and integration into clinical decision-making processes. This study highlights research gaps and challenges in understanding CERSs, encouraging further investigation by relevant studies and organizations. Lastly, the significance of meta-learning approaches in improving CERS performance and guiding future research endeavors is underscored.

Related papers

• EEG Emotion Copilot: Pruning LLMs for Emotional EEG Interpretation with Assisted Medical Record Generation [13.048477440429195]
  This paper presents the EEG Emotion Copilot, a system leveraging a lightweight large language model (LLM) operating in a local setting. The system is designed to first recognize emotional states directly from EEG signals, subsequently generate personalized diagnostic and treatment suggestions. Privacy concerns are also addressed, with a focus on ethical data collection, processing, and the protection of users' personal information.
  arXiv  Detail & Related papers  (2024-09-30T19:15:05Z)
• Multi-modal Mood Reader: Pre-trained Model Empowers Cross-Subject Emotion Recognition [23.505616142198487]
  We develop a Pre-trained model based Multimodal Mood Reader for cross-subject emotion recognition. The model learns universal latent representations of EEG signals through pre-training on large scale dataset. Extensive experiments on public datasets demonstrate Mood Reader's superior performance in cross-subject emotion recognition tasks.
  arXiv  Detail & Related papers  (2024-05-28T14:31:11Z)
• A Knowledge-Driven Cross-view Contrastive Learning for EEG Representation [48.85731427874065]
  This paper proposes a knowledge-driven cross-view contrastive learning framework (KDC2) to extract effective representations from EEG with limited labels. The KDC2 method creates scalp and neural views of EEG signals, simulating the internal and external representation of brain activity. By modeling prior neural knowledge based on neural information consistency theory, the proposed method extracts invariant and complementary neural knowledge to generate combined representations.
  arXiv  Detail & Related papers  (2023-09-21T08:53:51Z)
• Emotion recognition based on multi-modal electrophysiology multi-head attention Contrastive Learning [3.2536246345549538]
  We propose ME-MHACL, a self-supervised contrastive learning-based multimodal emotion recognition method. We apply the trained feature extractor to labeled electrophysiological signals and use multi-head attention mechanisms for feature fusion. Our method outperformed existing benchmark methods in emotion recognition tasks and had good cross-individual generalization ability.
  arXiv  Detail & Related papers  (2023-07-12T05:55:40Z)
• Synthesizing Affective Neurophysiological Signals Using Generative Models: A Review Paper [28.806992102323324]
  The integration of emotional intelligence in machines is an important step in advancing human-computer interaction. The scarcity of public affective datasets presents a challenge. We emphasize the use of generative models to address this issue in neurophysiological signals.
  arXiv  Detail & Related papers  (2023-06-05T08:38:30Z)
• fMRI from EEG is only Deep Learning away: the use of interpretable DL to unravel EEG-fMRI relationships [68.8204255655161]
  We present an interpretable domain grounded solution to recover the activity of several subcortical regions from multichannel EEG data. We recover individual spatial and time-frequency patterns of scalp EEG predictive of the hemodynamic signal in the subcortical nuclei.
  arXiv  Detail & Related papers  (2022-10-23T15:11:37Z)
• Multimodal Emotion Recognition using Transfer Learning from Speaker Recognition and BERT-based models [53.31917090073727]
  We propose a neural network-based emotion recognition framework that uses a late fusion of transfer-learned and fine-tuned models from speech and text modalities. We evaluate the effectiveness of our proposed multimodal approach on the interactive emotional dyadic motion capture dataset.
  arXiv  Detail & Related papers  (2022-02-16T00:23:42Z)
• Contrastive Learning of Subject-Invariant EEG Representations for Cross-Subject Emotion Recognition [9.07006689672858]
  We propose Contrast Learning method for Inter-Subject Alignment (ISA) for reliable cross-subject emotion recognition. ISA involves maximizing the similarity in EEG signals across subjects when they received the same stimuli in contrast to different ones. A convolutional neural network with depthwise spatial convolution and temporal convolution layers was applied to learn inter-subject representations from raw EEG signals.
  arXiv  Detail & Related papers  (2021-09-20T14:13:45Z)
• Stimuli-Aware Visual Emotion Analysis [75.68305830514007]
  We propose a stimuli-aware visual emotion analysis (VEA) method consisting of three stages, namely stimuli selection, feature extraction and emotion prediction. To the best of our knowledge, it is the first time to introduce stimuli selection process into VEA in an end-to-end network. Experiments demonstrate that the proposed method consistently outperforms the state-of-the-art approaches on four public visual emotion datasets.
  arXiv  Detail & Related papers  (2021-09-04T08:14:52Z)
• Continuous Emotion Recognition via Deep Convolutional Autoencoder and Support Vector Regressor [70.2226417364135]
  It is crucial that the machine should be able to recognize the emotional state of the user with high accuracy. Deep neural networks have been used with great success in recognizing emotions. We present a new model for continuous emotion recognition based on facial expression recognition.
  arXiv  Detail & Related papers  (2020-01-31T17:47:16Z)
• EEG-based Brain-Computer Interfaces (BCIs): A Survey of Recent Studies on Signal Sensing Technologies and Computational Intelligence Approaches and their Applications [65.32004302942218]
  Brain-Computer Interface (BCI) is a powerful communication tool between users and systems. Recent technological advances have increased interest in electroencephalographic (EEG) based BCI for translational and healthcare applications.
  arXiv  Detail & Related papers  (2020-01-28T10:36:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.