Graph Adapter of EEG Foundation Models for Parameter Efficient Fine Tuning

• URL: http://arxiv.org/abs/2411.16155v1
• Date: Mon, 25 Nov 2024 07:30:52 GMT
• Title: Graph Adapter of EEG Foundation Models for Parameter Efficient Fine Tuning
• Authors: Toyotaro Suzumura, Hiroki Kanezashi, Shotaro Akahori,
• Abstract summary: EEG-GraphAdapter (EGA) is a parameter-efficient fine-tuning (PEFT) approach to address these challenges. EGA is integrated into pre-trained temporal backbone models as a GNN-based module. It improves performance by up to 16.1% in the F1-score compared with the backbone BENDR model.
• Score: 1.8946099300030472
• License:
• Abstract: In diagnosing mental diseases from electroencephalography (EEG) data, neural network models such as Transformers have been employed to capture temporal dynamics. Additionally, it is crucial to learn the spatial relationships between EEG sensors, for which Graph Neural Networks (GNNs) are commonly used. However, fine-tuning large-scale complex neural network models simultaneously to capture both temporal and spatial features increases computational costs due to the more significant number of trainable parameters. It causes the limited availability of EEG datasets for downstream tasks, making it challenging to fine-tune large models effectively. We propose EEG-GraphAdapter (EGA), a parameter-efficient fine-tuning (PEFT) approach to address these challenges. EGA is integrated into pre-trained temporal backbone models as a GNN-based module and fine-tuned itself alone while keeping the backbone model parameters frozen. This enables the acquisition of spatial representations of EEG signals for downstream tasks, significantly reducing computational overhead and data requirements. Experimental evaluations on healthcare-related downstream tasks of Major Depressive Disorder and Abnormality Detection demonstrate that our EGA improves performance by up to 16.1% in the F1-score compared with the backbone BENDR model.

Related papers

• RISE-iEEG: Robust to Inter-Subject Electrodes Implantation Variability iEEG Classifier [0.0]
  RISE-iEEG stands for Robust Inter-Subject Electrode Implantation Variability iEEG. We developed an iEEG decoder model that can be applied across multiple patients' data without requiring the coordinates of electrode for each patient. Our analysis shows that the performance of RISE-iEEG is 10% higher than that of HTNet and EEGNet in terms of F1 score.
  arXiv  Detail & Related papers  (2024-08-12T18:33:19Z)
• Synthesizing Multimodal Electronic Health Records via Predictive Diffusion Models [69.06149482021071]
  We propose a novel EHR data generation model called EHRPD. It is a diffusion-based model designed to predict the next visit based on the current one while also incorporating time interval estimation. We conduct experiments on two public datasets and evaluate EHRPD from fidelity, privacy, and utility perspectives.
  arXiv  Detail & Related papers  (2024-06-20T02:20:23Z)
• REST: Efficient and Accelerated EEG Seizure Analysis through Residual State Updates [54.96885726053036]
  This paper introduces a novel graph-based residual state update mechanism (REST) for real-time EEG signal analysis. By leveraging a combination of graph neural networks and recurrent structures, REST efficiently captures both non-Euclidean geometry and temporal dependencies within EEG data. Our model demonstrates high accuracy in both seizure detection and classification tasks.
  arXiv  Detail & Related papers  (2024-06-03T16:30:19Z)
• Integrative Deep Learning Framework for Parkinson's Disease Early Detection using Gait Cycle Data Measured by Wearable Sensors: A CNN-GRU-GNN Approach [0.3222802562733786]
  We present a pioneering deep learning architecture tailored for the binary classification of subjects. Our model harnesses the power of 1D-Convolutional Neural Networks (CNN), Gated Recurrent Units (GRU), and Graph Neural Network (GNN) layers. Our proposed model achieves exceptional performance metrics, boasting accuracy, precision, recall, and F1 score values of 99.51%, 99.57%, 99.71%, and 99.64%, respectively.
  arXiv  Detail & Related papers  (2024-04-09T15:19:13Z)
• Neuro-GPT: Towards A Foundation Model for EEG [0.04188114563181615]
  We propose Neuro-GPT, a foundation model consisting of an EEG encoder and a GPT model. Foundation model is pre-trained on a large-scale data set using a self-supervised task that learns how to reconstruct masked EEG segments. Experiments demonstrate that applying a foundation model can significantly improve classification performance compared to a model trained from scratch.
  arXiv  Detail & Related papers  (2023-11-07T07:07:18Z)
• DGSD: Dynamical Graph Self-Distillation for EEG-Based Auditory Spatial Attention Detection [49.196182908826565]
  Auditory Attention Detection (AAD) aims to detect target speaker from brain signals in a multi-speaker environment. Current approaches primarily rely on traditional convolutional neural network designed for processing Euclidean data like images. This paper proposes a dynamical graph self-distillation (DGSD) approach for AAD, which does not require speech stimuli as input.
  arXiv  Detail & Related papers  (2023-09-07T13:43:46Z)
• Convolutional Monge Mapping Normalization for learning on sleep data [63.22081662149488]
  We propose a new method called Convolutional Monge Mapping Normalization (CMMN) CMMN consists in filtering the signals in order to adapt their power spectrum density (PSD) to a Wasserstein barycenter estimated on training data. Numerical experiments on sleep EEG data show that CMMN leads to significant and consistent performance gains independent from the neural network architecture.
  arXiv  Detail & Related papers  (2023-05-30T08:24:01Z)
• Brain Imaging-to-Graph Generation using Adversarial Hierarchical Diffusion Models for MCI Causality Analysis [44.45598796591008]
  Brain imaging-to-graph generation (BIGG) framework is proposed to map functional magnetic resonance imaging (fMRI) into effective connectivity for mild cognitive impairment analysis. The hierarchical transformers in the generator are designed to estimate the noise at multiple scales. Evaluations of the ADNI dataset demonstrate the feasibility and efficacy of the proposed model.
  arXiv  Detail & Related papers  (2023-05-18T06:54:56Z)
• Conditional Generative Models for Simulation of EMG During Naturalistic Movements [45.698312905115955]
  We present a conditional generative neural network trained adversarially to generate motor unit activation potential waveforms. We demonstrate the ability of such a model to predictively interpolate between a much smaller number of numerical model's outputs with a high accuracy.
  arXiv  Detail & Related papers  (2022-11-03T14:49:02Z)
• Efficient ECG-based Atrial Fibrillation Detection via Parameterised Hypercomplex Neural Networks [11.964843902569925]
  Atrial fibrillation (AF) is the most common cardiac arrhythmia and associated with a high risk for serious conditions like stroke. Wearable devices embedded with automatic and timely AF assessment from electrocardiograms (ECGs) has shown to be promising in preventing life-threatening situations. Deep neural networks have demonstrated superiority in model performance, their use on wearable devices is limited by the trade-off between model performance and complexity.
  arXiv  Detail & Related papers  (2022-10-27T14:24:48Z)
• Atrial Fibrillation Detection Using Weight-Pruned, Log-Quantised Convolutional Neural Networks [25.160063477248904]
  A convolutional neural network model is developed for detecting atrial fibrillation from electrocardiogram signals. The model demonstrates high performance despite being trained on limited, variable-length input data. The final model achieved a 91.1% model compression ratio while maintaining high model accuracy of 91.7% and less than 1% loss.
  arXiv  Detail & Related papers  (2022-06-14T11:47:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.