An EEG-Based Multi-Modal Emotion Database with Both Posed and Authentic Facial Actions for Emotion Analysis

• URL: http://arxiv.org/abs/2203.15829v1
• Date: Tue, 29 Mar 2022 18:02:12 GMT
• Title: An EEG-Based Multi-Modal Emotion Database with Both Posed and Authentic Facial Actions for Emotion Analysis
• Authors: Xiaotian Li, Xiang Zhang, Huiyuan Yang, Wenna Duan, Weiying Dai and Lijun Yin
• Abstract summary: We propose to develop a new database by collecting facial expressions, action units, and EEGs simultaneously. The database will be released to the research community to advance the state of the art for automatic emotion recognition.
• Score: 13.448832812713501
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Emotion is an experience associated with a particular pattern of physiological activity along with different physiological, behavioral and cognitive changes. One behavioral change is facial expression, which has been studied extensively over the past few decades. Facial behavior varies with a person's emotion according to differences in terms of culture, personality, age, context, and environment. In recent years, physiological activities have been used to study emotional responses. A typical signal is the electroencephalogram (EEG), which measures brain activity. Most of existing EEG-based emotion analysis has overlooked the role of facial expression changes. There exits little research on the relationship between facial behavior and brain signals due to the lack of dataset measuring both EEG and facial action signals simultaneously. To address this problem, we propose to develop a new database by collecting facial expressions, action units, and EEGs simultaneously. We recorded the EEGs and face videos of both posed facial actions and spontaneous expressions from 29 participants with different ages, genders, ethnic backgrounds. Differing from existing approaches, we designed a protocol to capture the EEG signals by evoking participants' individual action units explicitly. We also investigated the relation between the EEG signals and facial action units. As a baseline, the database has been evaluated through the experiments on both posed and spontaneous emotion recognition with images alone, EEG alone, and EEG fused with images, respectively. The database will be released to the research community to advance the state of the art for automatic emotion recognition.

Related papers

• Smile on the Face, Sadness in the Eyes: Bridging the Emotion Gap with a Multimodal Dataset of Eye and Facial Behaviors [49.833812625518554]
  We introduce eye behaviors as an important emotional cue and construct an Eye-behavior-aided Multimodal Emotion Recognition dataset.<n>In the experiment, we introduce seven multimodal benchmark protocols for a variety of comprehensive evaluations of the EMER dataset.<n>The results show that the EMERT outperforms other state-of-the-art multimodal methods by a great margin, revealing the importance of modeling eye behaviors for robust ER.
  arXiv  Detail & Related papers  (2025-12-18T12:52:55Z)
• Exploring the correlation between the type of music and the emotions evoked: A study using subjective questionnaires and EEG [51.04304650881465]
  The aim is to demonstrate the impact of different music genres on emotions.<n>This had the effect of capturing a wide range of emotional responses to music.<n>The analysis revealed connections between emotions and observed brain activity.
  arXiv  Detail & Related papers  (2025-10-30T09:43:56Z)
• CAST-Phys: Contactless Affective States Through Physiological signals Database [74.28082880875368]
  The lack of affective multi-modal datasets remains a major bottleneck in developing accurate emotion recognition systems.<n>We present the Contactless Affective States Through Physiological Signals Database (CAST-Phys), a novel high-quality dataset capable of remote physiological emotion recognition.<n>Our analysis highlights the crucial role of physiological signals in realistic scenarios where facial expressions alone may not provide sufficient emotional information.
  arXiv  Detail & Related papers  (2025-07-08T15:20:24Z)
• Disentangle Identity, Cooperate Emotion: Correlation-Aware Emotional Talking Portrait Generation [63.94836524433559]
  DICE-Talk is a framework for disentangling identity with emotion and cooperating emotions with similar characteristics. We develop a disentangled emotion embedder that jointly models audio-visual emotional cues through cross-modal attention. Second, we introduce a correlation-enhanced emotion conditioning module with learnable Emotion Banks. Third, we design an emotion discrimination objective that enforces affective consistency during the diffusion process.
  arXiv  Detail & Related papers  (2025-04-25T05:28:21Z)
• Smile upon the Face but Sadness in the Eyes: Emotion Recognition based on Facial Expressions and Eye Behaviors [63.194053817609024]
  We introduce eye behaviors as an important emotional cues for the creation of a new Eye-behavior-aided Multimodal Emotion Recognition dataset. For the first time, we provide annotations for both Emotion Recognition (ER) and Facial Expression Recognition (FER) in the EMER dataset. We specifically design a new EMERT architecture to concurrently enhance performance in both ER and FER.
  arXiv  Detail & Related papers  (2024-11-08T04:53:55Z)
• Emotion Analysis on EEG Signal Using Machine Learning and Neural Network [0.0]
  The main purpose of this study is to improve ways to improve emotion recognition performance using brain signals. Various approaches to human-machine interaction technologies have been ongoing for a long time, and in recent years, researchers have had great success in automatically understanding emotion using brain signals.
  arXiv  Detail & Related papers  (2023-07-09T09:50:34Z)
• Emotion Separation and Recognition from a Facial Expression by Generating the Poker Face with Vision Transformers [57.1091606948826]
  We propose a novel FER model, named Poker Face Vision Transformer or PF-ViT, to address these challenges. PF-ViT aims to separate and recognize the disturbance-agnostic emotion from a static facial image via generating its corresponding poker face. PF-ViT utilizes vanilla Vision Transformers, and its components are pre-trained as Masked Autoencoders on a large facial expression dataset.
  arXiv  Detail & Related papers  (2022-07-22T13:39:06Z)
• A Naturalistic Database of Thermal Emotional Facial Expressions and Effects of Induced Emotions on Memory [1.8352113484137629]
  This work defines a procedure for collecting naturally induced emotional facial expressions through the vision of movie excerpts with high emotional contents. The induced emotional states include the four basic emotions of sadness, disgust, happiness, and surprise. The resulting database contains both thermal and visible emotional facial expressions.
  arXiv  Detail & Related papers  (2022-03-29T11:17:35Z)
• Progressive Graph Convolution Network for EEG Emotion Recognition [35.08010382523394]
  Studies in the area of neuroscience have revealed the relationship between emotional patterns and brain functional regions. In EEG emotion recognition, we can observe that clearer boundaries exist between coarse-grained emotions than those between fine-grained emotions. We propose a progressive graph convolution network (PGCN) for capturing this inherent characteristic in EEG emotional signals.
  arXiv  Detail & Related papers  (2021-12-14T03:30:13Z)
• A Circular-Structured Representation for Visual Emotion Distribution Learning [82.89776298753661]
  We propose a well-grounded circular-structured representation to utilize the prior knowledge for visual emotion distribution learning. To be specific, we first construct an Emotion Circle to unify any emotional state within it. On the proposed Emotion Circle, each emotion distribution is represented with an emotion vector, which is defined with three attributes.
  arXiv  Detail & Related papers  (2021-06-23T14:53:27Z)
• Emotion pattern detection on facial videos using functional statistics [62.997667081978825]
  We propose a technique based on Functional ANOVA to extract significant patterns of face muscles movements. We determine if there are time-related differences on expressions among emotional groups by using a functional F-test.
  arXiv  Detail & Related papers  (2021-03-01T08:31:08Z)
• A Novel Transferability Attention Neural Network Model for EEG Emotion Recognition [51.203579838210885]
  We propose a transferable attention neural network (TANN) for EEG emotion recognition. TANN learns the emotional discriminative information by highlighting the transferable EEG brain regions data and samples adaptively. This can be implemented by measuring the outputs of multiple brain-region-level discriminators and one single sample-level discriminator.
  arXiv  Detail & Related papers  (2020-09-21T02:42:30Z)
• Impact of multiple modalities on emotion recognition: investigation into 3d facial landmarks, action units, and physiological data [4.617405932149653]
  We analyze 3D facial data, action units, and physiological data as it relates to their impact on emotion recognition. Our analysis indicates that both 3D facial landmarks and physiological data are encouraging for expression/emotion recognition. On the other hand, while action units can positively impact emotion recognition when fused with other modalities, the results suggest it is difficult to detect emotion using them in a unimodal fashion.
  arXiv  Detail & Related papers  (2020-05-17T18:59:57Z)
• Emotion Recognition From Gait Analyses: Current Research and Future Directions [48.93172413752614]
  gait conveys information about the walker's emotion. The mapping between various emotions and gait patterns provides a new source for automated emotion recognition. gait is remotely observable, more difficult to imitate, and requires less cooperation from the subject.
  arXiv  Detail & Related papers  (2020-03-13T08:22:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.