Graph Adapter of EEG Foundation Models for Parameter Efficient Fine Tuning

• URL: http://arxiv.org/abs/2411.16155v1
• Date: Mon, 25 Nov 2024 07:30:52 GMT
• Title: Graph Adapter of EEG Foundation Models for Parameter Efficient Fine Tuning
• Authors: Toyotaro Suzumura, Hiroki Kanezashi, Shotaro Akahori,
• Abstract summary: EEG-GraphAdapter (EGA) is a parameter-efficient fine-tuning (PEFT) approach to address these challenges. EGA is integrated into pre-trained temporal backbone models as a GNN-based module. It improves performance by up to 16.1% in the F1-score compared with the backbone BENDR model.
• Score: 1.8946099300030472
• License:
• Abstract: In diagnosing mental diseases from electroencephalography (EEG) data, neural network models such as Transformers have been employed to capture temporal dynamics. Additionally, it is crucial to learn the spatial relationships between EEG sensors, for which Graph Neural Networks (GNNs) are commonly used. However, fine-tuning large-scale complex neural network models simultaneously to capture both temporal and spatial features increases computational costs due to the more significant number of trainable parameters. It causes the limited availability of EEG datasets for downstream tasks, making it challenging to fine-tune large models effectively. We propose EEG-GraphAdapter (EGA), a parameter-efficient fine-tuning (PEFT) approach to address these challenges. EGA is integrated into pre-trained temporal backbone models as a GNN-based module and fine-tuned itself alone while keeping the backbone model parameters frozen. This enables the acquisition of spatial representations of EEG signals for downstream tasks, significantly reducing computational overhead and data requirements. Experimental evaluations on healthcare-related downstream tasks of Major Depressive Disorder and Abnormality Detection demonstrate that our EGA improves performance by up to 16.1% in the F1-score compared with the backbone BENDR model.

Related papers

• FEMBA: Efficient and Scalable EEG Analysis with a Bidirectional Mamba Foundation Model [19.91895489891802]
  We present FEMBA (Foundational EEG Mamba + Bidirectional Architecture), a novel self-supervised framework for EEG analysis. Unlike Transformer-based models, which incur quadratic time and memory complexity, FEMBA scales linearly with sequence length. It achieves competitive performance in comparison with transformer models, with significantly lower computational cost.
  arXiv  Detail & Related papers  (2025-02-10T13:15:52Z)
• CEReBrO: Compact Encoder for Representations of Brain Oscillations Using Efficient Alternating Attention [53.539020807256904]
  We introduce a Compact for Representations of Brain Oscillations using alternating attention (CEReBrO) Our tokenization scheme represents EEG signals at a per-channel patch. We propose an alternating attention mechanism that jointly models intra-channel temporal dynamics and inter-channel spatial correlations, achieving 2x speed improvement with 6x less memory required compared to standard self-attention.
  arXiv  Detail & Related papers  (2025-01-18T21:44:38Z)
• Graph-Enhanced EEG Foundation Model [16.335330142000657]
  We propose a novel foundation model for EEG that integrates both temporal and inter-channel information. Our architecture combines Graph Neural Networks (GNNs), which effectively capture relational structures, with a masked autoencoder to enable efficient pre-training.
  arXiv  Detail & Related papers  (2024-11-29T06:57:50Z)
• RISE-iEEG: Robust to Inter-Subject Electrodes Implantation Variability iEEG Classifier [0.0]
  RISE-iEEG stands for Robust Inter-Subject Electrode Implantation Variability iEEG. We developed an iEEG decoder model that can be applied across multiple patients' data without requiring the coordinates of electrode for each patient. Our analysis shows that the performance of RISE-iEEG is 10% higher than that of HTNet and EEGNet in terms of F1 score.
  arXiv  Detail & Related papers  (2024-08-12T18:33:19Z)
• Synthesizing Multimodal Electronic Health Records via Predictive Diffusion Models [69.06149482021071]
  We propose a novel EHR data generation model called EHRPD. It is a diffusion-based model designed to predict the next visit based on the current one while also incorporating time interval estimation. We conduct experiments on two public datasets and evaluate EHRPD from fidelity, privacy, and utility perspectives.
  arXiv  Detail & Related papers  (2024-06-20T02:20:23Z)
• REST: Efficient and Accelerated EEG Seizure Analysis through Residual State Updates [54.96885726053036]
  This paper introduces a novel graph-based residual state update mechanism (REST) for real-time EEG signal analysis. By leveraging a combination of graph neural networks and recurrent structures, REST efficiently captures both non-Euclidean geometry and temporal dependencies within EEG data. Our model demonstrates high accuracy in both seizure detection and classification tasks.
  arXiv  Detail & Related papers  (2024-06-03T16:30:19Z)
• hvEEGNet: exploiting hierarchical VAEs on EEG data for neuroscience applications [3.031375888004876]
  Two main issues challenge the existing DL-based modeling methods for EEG. High variability between subjects and low signal-to-noise ratio make it difficult to ensure a good quality in the EEG data. We propose two variational autoencoder models, namely vEEGNet-ver3 and hvEEGNet, to target the problem of high-fidelity EEG reconstruction.
  arXiv  Detail & Related papers  (2023-11-20T15:36:31Z)
• Neuro-GPT: Towards A Foundation Model for EEG [0.04188114563181615]
  We propose Neuro-GPT, a foundation model consisting of an EEG encoder and a GPT model. Foundation model is pre-trained on a large-scale data set using a self-supervised task that learns how to reconstruct masked EEG segments. Experiments demonstrate that applying a foundation model can significantly improve classification performance compared to a model trained from scratch.
  arXiv  Detail & Related papers  (2023-11-07T07:07:18Z)
• DGSD: Dynamical Graph Self-Distillation for EEG-Based Auditory Spatial Attention Detection [49.196182908826565]
  Auditory Attention Detection (AAD) aims to detect target speaker from brain signals in a multi-speaker environment. Current approaches primarily rely on traditional convolutional neural network designed for processing Euclidean data like images. This paper proposes a dynamical graph self-distillation (DGSD) approach for AAD, which does not require speech stimuli as input.
  arXiv  Detail & Related papers  (2023-09-07T13:43:46Z)
• EEG-Inception: An Accurate and Robust End-to-End Neural Network for EEG-based Motor Imagery Classification [123.93460670568554]
  This paper proposes a novel convolutional neural network (CNN) architecture for accurate and robust EEG-based motor imagery (MI) classification. The proposed CNN model, namely EEG-Inception, is built on the backbone of the Inception-Time network. The proposed network is an end-to-end classification, as it takes the raw EEG signals as the input and does not require complex EEG signal-preprocessing.
  arXiv  Detail & Related papers  (2021-01-24T19:03:10Z)
• ECG-DelNet: Delineation of Ambulatory Electrocardiograms with Mixed Quality Labeling Using Neural Networks [69.25956542388653]
  Deep learning (DL) algorithms are gaining weight in academic and industrial settings. We demonstrate DL can be successfully applied to low interpretative tasks by embedding ECG detection and delineation onto a segmentation framework. The model was trained using PhysioNet's QT database, comprised of 105 ambulatory ECG recordings.
  arXiv  Detail & Related papers  (2020-05-11T16:29:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.