EEG Emotion Copilot: Optimizing Lightweight LLMs for Emotional EEG Interpretation with Assisted Medical Record Generation

• URL: http://arxiv.org/abs/2410.00166v2
• Date: Tue, 07 Jan 2025 03:21:43 GMT
• Title: EEG Emotion Copilot: Optimizing Lightweight LLMs for Emotional EEG Interpretation with Assisted Medical Record Generation
• Authors: Hongyu Chen, Weiming Zeng, Chengcheng Chen, Luhui Cai, Fei Wang, Yuhu Shi, Lei Wang, Wei Zhang, Yueyang Li, Hongjie Yan, Wai Ting Siok, Nizhuan Wang,
• Abstract summary: This paper presents the EEG Emotion Copilot, which first recognizes emotional states directly from EEG signals. It then generates personalized diagnostic and treatment suggestions, and finally supports the automation of assisted electronic medical records. The proposed copilot is expected to advance the application of affective computing in the medical domain.
• Score: 12.707059419820848
• License:
• Abstract: In the fields of affective computing (AC) and brain-machine interface (BMI), the analysis of physiological and behavioral signals to discern individual emotional states has emerged as a critical research frontier. While deep learning-based approaches have made notable strides in EEG emotion recognition, particularly in feature extraction and pattern recognition, significant challenges persist in achieving end-to-end emotion computation, including real-time processing, individual adaptation, and seamless user interaction. This paper presents the EEG Emotion Copilot, a system optimizing a lightweight large language model (LLM) with 0.5B parameters operating in a local setting, which first recognizes emotional states directly from EEG signals, subsequently generates personalized diagnostic and treatment suggestions, and finally supports the automation of assisted electronic medical records. Specifically, we demonstrate the critical techniques in the novel data structure of prompt, model pruning and fine-tuning training, and deployment strategies aiming at improving real-time performance and computational efficiency. Extensive experiments show that our optimized lightweight LLM-based copilot achieves an enhanced intuitive interface for participant interaction, superior accuracy of emotion recognition and assisted electronic medical records generation, in comparison to such models with similar scale parameters or large-scale parameters such as 1.5B, 1.8B, 3B and 7B. In summary, through these efforts, the proposed copilot is expected to advance the application of AC in the medical domain, offering innovative solution to mental health monitoring. The codes will be released at https://github.com/NZWANG/EEG_Emotion_Copilot.

Related papers

• Hybrid Quantum Deep Learning Model for Emotion Detection using raw EEG Signal Analysis [0.0]
  This work presents a hybrid quantum deep learning technique for emotion recognition. Conventional EEG-based emotion recognition techniques are limited by noise and high-dimensional data complexity. The model will be extended for real-time applications and multi-class categorization in future study.
  arXiv  Detail & Related papers  (2024-11-19T17:44:04Z)
• Smile upon the Face but Sadness in the Eyes: Emotion Recognition based on Facial Expressions and Eye Behaviors [63.194053817609024]
  We introduce eye behaviors as an important emotional cues for the creation of a new Eye-behavior-aided Multimodal Emotion Recognition dataset. For the first time, we provide annotations for both Emotion Recognition (ER) and Facial Expression Recognition (FER) in the EMER dataset. We specifically design a new EMERT architecture to concurrently enhance performance in both ER and FER.
  arXiv  Detail & Related papers  (2024-11-08T04:53:55Z)
• Complex Emotion Recognition System using basic emotions via Facial Expression, EEG, and ECG Signals: a review [1.8310098790941458]
  The Complex Emotion Recognition System (CERS) deciphers complex emotional states by examining combinations of basic emotions expressed, their interconnections, and the dynamic variations. The development of AI systems for discerning complex emotions poses a substantial challenge with significant implications for affective computing. incorporating physiological signals such as Electrocardiogram (ECG) and Electroencephalogram (EEG) can notably enhance CERS.
  arXiv  Detail & Related papers  (2024-09-09T05:06:10Z)
• Emotion-Agent: Unsupervised Deep Reinforcement Learning with Distribution-Prototype Reward for Continuous Emotional EEG Analysis [2.1645626994550664]
  Continuous electroencephalography (EEG) signals are widely used in affective brain-computer interface (aBCI) applications. We propose a novel unsupervised deep reinforcement learning framework, called Emotion-Agent, to automatically identify relevant and informative emotional moments from EEG signals. Emotion-Agent is trained using Proximal Policy Optimization (PPO) to achieve stable and efficient convergence.
  arXiv  Detail & Related papers  (2024-08-22T04:29:25Z)
• Multi-modal Mood Reader: Pre-trained Model Empowers Cross-Subject Emotion Recognition [23.505616142198487]
  We develop a Pre-trained model based Multimodal Mood Reader for cross-subject emotion recognition. The model learns universal latent representations of EEG signals through pre-training on large scale dataset. Extensive experiments on public datasets demonstrate Mood Reader's superior performance in cross-subject emotion recognition tasks.
  arXiv  Detail & Related papers  (2024-05-28T14:31:11Z)
• Multimodal Emotion Recognition using Transfer Learning from Speaker Recognition and BERT-based models [53.31917090073727]
  We propose a neural network-based emotion recognition framework that uses a late fusion of transfer-learned and fine-tuned models from speech and text modalities. We evaluate the effectiveness of our proposed multimodal approach on the interactive emotional dyadic motion capture dataset.
  arXiv  Detail & Related papers  (2022-02-16T00:23:42Z)
• Cross-individual Recognition of Emotions by a Dynamic Entropy based on Pattern Learning with EEG features [2.863100352151122]
  We propose a deep-learning framework denoted as a dynamic entropy-based pattern learning (DEPL) to abstract informative indicators pertaining to the neurophysiological features among multiple individuals. DEPL enhanced the capability of representations generated by a deep convolutional neural network by modelling the interdependencies between the cortical locations of dynamical entropy based features.
  arXiv  Detail & Related papers  (2020-09-26T07:22:07Z)
• A Novel Transferability Attention Neural Network Model for EEG Emotion Recognition [51.203579838210885]
  We propose a transferable attention neural network (TANN) for EEG emotion recognition. TANN learns the emotional discriminative information by highlighting the transferable EEG brain regions data and samples adaptively. This can be implemented by measuring the outputs of multiple brain-region-level discriminators and one single sample-level discriminator.
  arXiv  Detail & Related papers  (2020-09-21T02:42:30Z)
• Uncovering the structure of clinical EEG signals with self-supervised learning [64.4754948595556]
  Supervised learning paradigms are often limited by the amount of labeled data that is available. This phenomenon is particularly problematic in clinically-relevant data, such as electroencephalography (EEG) By extracting information from unlabeled data, it might be possible to reach competitive performance with deep neural networks.
  arXiv  Detail & Related papers  (2020-07-31T14:34:47Z)
• Continuous Emotion Recognition via Deep Convolutional Autoencoder and Support Vector Regressor [70.2226417364135]
  It is crucial that the machine should be able to recognize the emotional state of the user with high accuracy. Deep neural networks have been used with great success in recognizing emotions. We present a new model for continuous emotion recognition based on facial expression recognition.
  arXiv  Detail & Related papers  (2020-01-31T17:47:16Z)
• EEG-based Brain-Computer Interfaces (BCIs): A Survey of Recent Studies on Signal Sensing Technologies and Computational Intelligence Approaches and their Applications [65.32004302942218]
  Brain-Computer Interface (BCI) is a powerful communication tool between users and systems. Recent technological advances have increased interest in electroencephalographic (EEG) based BCI for translational and healthcare applications.
  arXiv  Detail & Related papers  (2020-01-28T10:36:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.