FEMBA: Efficient and Scalable EEG Analysis with a Bidirectional Mamba Foundation Model

• URL: http://arxiv.org/abs/2502.06438v1
• Date: Mon, 10 Feb 2025 13:15:52 GMT
• Title: FEMBA: Efficient and Scalable EEG Analysis with a Bidirectional Mamba Foundation Model
• Authors: Anna Tegon, Thorir Mar Ingolfsson, Xiaying Wang, Luca Benini, Yawei Li,
• Abstract summary: We present FEMBA (Foundational EEG Mamba + Bidirectional Architecture), a novel self-supervised framework for EEG analysis.<n>Unlike Transformer-based models, which incur quadratic time and memory complexity, FEMBA scales linearly with sequence length.<n>It achieves competitive performance in comparison with transformer models, with significantly lower computational cost.
• Score: 19.91895489891802
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Accurate and efficient electroencephalography (EEG) analysis is essential for detecting seizures and artifacts in long-term monitoring, with applications spanning hospital diagnostics to wearable health devices. Robust EEG analytics have the potential to greatly improve patient care. However, traditional deep learning models, especially Transformer-based architectures, are hindered by their quadratic time and memory complexity, making them less suitable for resource-constrained environments. To address these challenges, we present FEMBA (Foundational EEG Mamba + Bidirectional Architecture), a novel self-supervised framework that establishes new efficiency benchmarks for EEG analysis through bidirectional state-space modeling. Unlike Transformer-based models, which incur quadratic time and memory complexity, FEMBA scales linearly with sequence length, enabling more scalable and efficient processing of extended EEG recordings. Trained on over 21,000 hours of unlabeled EEG and fine-tuned on three downstream tasks, FEMBA achieves competitive performance in comparison with transformer models, with significantly lower computational cost. Specifically, it reaches 81.82% balanced accuracy (0.8921 AUROC) on TUAB and 0.949 AUROC on TUAR, while a tiny 7.8M-parameter variant demonstrates viability for resource-constrained devices. These results pave the way for scalable, general-purpose EEG analytics in both clinical and highlight FEMBA as a promising candidate for wearable applications.

Related papers

• BioSerenity-E1: a self-supervised EEG model for medical applications [0.0]
  BioSerenity-E1 is a family of self-supervised foundation models for clinical EEG applications. It combines spectral tokenization with masked prediction to achieve state-of-the-art performance across relevant diagnostic tasks.
  arXiv  Detail & Related papers  (2025-03-13T13:42:46Z)
• rECGnition_v2.0: Self-Attentive Canonical Fusion of ECG and Patient Data using deep learning for effective Cardiac Diagnostics [0.56337958460022]
  This study uses MIT-BIH Arrhythmia dataset to evaluate the efficiency of rECGnition_v2.0 for various classes of arrhythmias. The compact architectural footprint of the rECGnition_v2.0, characterized by its lesser trainable parameters, unfurled several advantages including interpretability and scalability.
  arXiv  Detail & Related papers  (2025-02-22T15:16:46Z)
• Finetuning and Quantization of EEG-Based Foundational BioSignal Models on ECG and PPG Data for Blood Pressure Estimation [53.2981100111204]
  Photoplethysmography and electrocardiography can potentially enable continuous blood pressure (BP) monitoring. Yet accurate and robust machine learning (ML) models remains challenging due to variability in data quality and patient-specific factors. In this work, we investigate whether a model pre-trained on one modality can effectively be exploited to improve the accuracy of a different signal type. Our approach achieves near state-of-the-art accuracy for diastolic BP and surpasses by 1.5x the accuracy of prior works for systolic BP.
  arXiv  Detail & Related papers  (2025-02-10T13:33:12Z)
• CEReBrO: Compact Encoder for Representations of Brain Oscillations Using Efficient Alternating Attention [53.539020807256904]
  We introduce a Compact for Representations of Brain Oscillations using alternating attention (CEReBrO)<n>Our tokenization scheme represents EEG signals at a per-channel patch.<n>We propose an alternating attention mechanism that jointly models intra-channel temporal dynamics and inter-channel spatial correlations, achieving 2x speed improvement with 6x less memory required compared to standard self-attention.
  arXiv  Detail & Related papers  (2025-01-18T21:44:38Z)
• CwA-T: A Channelwise AutoEncoder with Transformer for EEG Abnormality Detection [0.4448543797168715]
  CwA-T is a novel framework that combines a channelwise CNN-based autoencoder with a single-head transformer classifier for efficient EEG abnormality detection.<n> evaluated on the TUH Abnormal EEG Corpus, the proposed model achieves 85.0% accuracy, 76.2% sensitivity, and 91.2% specificity at the per-case level.<n>The framework retains interpretability through its channelwise design, demonstrating great potential for future applications in neuroscience research and clinical practice.
  arXiv  Detail & Related papers  (2024-12-19T04:38:34Z)
• Graph Adapter of EEG Foundation Models for Parameter Efficient Fine Tuning [1.8946099300030472]
  EEG-GraphAdapter (EGA) is a parameter-efficient fine-tuning (PEFT) approach to address these challenges. EGA is integrated into pre-trained temporal backbone models as a GNN-based module. It improves performance by up to 16.1% in the F1-score compared with the backbone BENDR model.
  arXiv  Detail & Related papers  (2024-11-25T07:30:52Z)
• FoME: A Foundation Model for EEG using Adaptive Temporal-Lateral Attention Scaling [19.85701025524892]
  FoME (Foundation Model for EEG) is a novel approach using adaptive temporal-lateral attention scaling. FoME is pre-trained on a diverse 1.7TB dataset of scalp and intracranial EEG recordings, comprising 745M parameters trained for 1,096k steps.
  arXiv  Detail & Related papers  (2024-09-19T04:22:40Z)
• EEGMobile: Enhancing Speed and Accuracy in EEG-Based Gaze Prediction with Advanced Mobile Architectures [0.36832029288386137]
  This study presents a model that leverages a pre-trained MobileViT alongside Knowledge Distillation (KD) for EEG regression tasks. Our results showcase that this model is capable of performing at a level comparable (only 3% lower) to the previous State-Of-The-Art (SOTA) on the EEGEyeNet Absolute Position Task.
  arXiv  Detail & Related papers  (2024-08-06T21:02:27Z)
• REST: Efficient and Accelerated EEG Seizure Analysis through Residual State Updates [54.96885726053036]
  This paper introduces a novel graph-based residual state update mechanism (REST) for real-time EEG signal analysis. By leveraging a combination of graph neural networks and recurrent structures, REST efficiently captures both non-Euclidean geometry and temporal dependencies within EEG data. Our model demonstrates high accuracy in both seizure detection and classification tasks.
  arXiv  Detail & Related papers  (2024-06-03T16:30:19Z)
• EKGNet: A 10.96{\mu}W Fully Analog Neural Network for Intra-Patient Arrhythmia Classification [79.7946379395238]
  We present an integrated approach by combining analog computing and deep learning for electrocardiogram (ECG) arrhythmia classification. We propose EKGNet, a hardware-efficient and fully analog arrhythmia classification architecture that archives high accuracy with low power consumption.
  arXiv  Detail & Related papers  (2023-10-24T02:37:49Z)
• Multiple Time Series Fusion Based on LSTM An Application to CAP A Phase Classification Using EEG [56.155331323304]
  Deep learning based electroencephalogram channels' feature level fusion is carried out in this work. Channel selection, fusion, and classification procedures were optimized by two optimization algorithms.
  arXiv  Detail & Related papers  (2021-12-18T14:17:49Z)
• EEG-Inception: An Accurate and Robust End-to-End Neural Network for EEG-based Motor Imagery Classification [123.93460670568554]
  This paper proposes a novel convolutional neural network (CNN) architecture for accurate and robust EEG-based motor imagery (MI) classification. The proposed CNN model, namely EEG-Inception, is built on the backbone of the Inception-Time network. The proposed network is an end-to-end classification, as it takes the raw EEG signals as the input and does not require complex EEG signal-preprocessing.
  arXiv  Detail & Related papers  (2021-01-24T19:03:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.