Mentality: A Mamba-based Approach towards Foundation Models for EEG

• URL: http://arxiv.org/abs/2509.02746v1
• Date: Tue, 02 Sep 2025 18:47:38 GMT
• Title: Mentality: A Mamba-based Approach towards Foundation Models for EEG
• Authors: Saarang Panchavati, Corey Arnold, William Speier,
• Abstract summary: This study explores the potential of foundation models, specifically a Mamba-based selective state space model, for enhancing EEG analysis in neurological disorder diagnosis.<n>By training a Mamba-based model on a large dataset containing seizure and non-seizure EEG recordings, we demonstrate the model's effectiveness, achieving an AUROC of 0.72 on a held-out test set.
• Score: 3.263390674277623
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This work explores the potential of foundation models, specifically a Mamba-based selective state space model, for enhancing EEG analysis in neurological disorder diagnosis. EEG, crucial for diagnosing conditions like epilepsy, presents significant challenges due to its noisy, high-dimensional, and nonlinear nature. Traditional machine learning methods have made advances in automating EEG analysis but often fail to capture its complex spatio-temporal dynamics. Recent advances in deep learning, particularly in sequence modeling, offer new avenues for creating more generalized and expressive models capable of handling such complexities. By training a Mamba-based model on a large dataset containing seizure and non-seizure EEG recordings through a self-supervised reconstruction task followed by a seizure detection task, we demonstrate the model's effectiveness, achieving an AUROC of 0.72 on a held-out test set. This approach marks a significant step toward developing large-scale, clinically applicable foundation models for EEG data analysis.

Related papers

• Investigating the Impact of Histopathological Foundation Models on Regressive Prediction of Homologous Recombination Deficiency [52.50039435394964]
  We systematically evaluate foundation models for regression-based tasks.<n>We extract patch-level features from whole slide images (WSI) using five state-of-the-art foundation models.<n>Models are trained to predict continuous HRD scores based on these extracted features across breast, endometrial, and lung cancer cohorts.
  arXiv  Detail & Related papers  (2026-01-29T14:06:50Z)
• Counterfactual Probabilistic Diffusion with Expert Models [47.31408854040995]
  We propose a time series diffusion-based framework that incorporates guidance from imperfect expert models.<n>Our method, ODE-Diff, bridges mechanistic and data-driven approaches, enabling more reliable and interpretable causal inference.
  arXiv  Detail & Related papers  (2025-08-18T20:44:32Z)
• Towards Generalizable Learning Models for EEG-Based Identification of Pain Perception [1.718323575065371]
  We systematically evaluate the performance of cross-participant generalization of a wide range of machine learning models.<n>Traditional models suffered the largest drop from within- to cross-participant performance, while deep learning models proved more resilient.<n>Even though performance variability remained high, the strong results of the graph-based model highlight its potential to capture subject-invariant structure in EEG signals.
  arXiv  Detail & Related papers  (2025-08-12T09:57:32Z)
• NeuroDx-LM: A Clinical Large-Scale Model for EEG-based Neurological Disorder Detection [7.185477956123345]
  Large-scale models pre-trained on Electroencephalography (EEG) have shown promise in clinical applications such as neurological disorder detection.<n>NeuroDx-LM is a novel large-scale model specifically designed for detecting EEG-based neurological disorders.
  arXiv  Detail & Related papers  (2025-08-11T16:02:25Z)
• A Vector-Quantized Foundation Model for Patient Behavior Monitoring [43.02353546717171]
  This paper introduces a novel foundation model based on a modified vector quantized variational autoencoder, specifically designed to process real-world data from smartphones and wearable devices.<n>We leveraged the discrete latent representation of this model to effectively perform two downstream tasks, suicide risk assessment and emotional state prediction, on different held-out clinical cohorts without the need of fine-tuning.
  arXiv  Detail & Related papers  (2025-03-19T14:01:16Z)
• Are foundation models useful feature extractors for electroencephalography analysis? [9.413178499853156]
  We investigate the effectiveness of foundation models in medical time series analysis involving electroencephalography (EEG)<n>Our analysis shows that foundation models extract meaningful EEG features, outperform specialised models even without domain adaptation, and localise task-specific biomarkers.
  arXiv  Detail & Related papers  (2025-02-28T14:21:34Z)
• Large Cognition Model: Towards Pretrained EEG Foundation Model [0.0]
  We propose a transformer-based foundation model designed to generalize across diverse EEG datasets and downstream tasks.<n>Our findings highlight the potential of pretrained EEG foundation models to accelerate advancements in neuroscience, personalized medicine, and BCI technology.
  arXiv  Detail & Related papers  (2025-02-11T04:28:10Z)
• Recent Advances in Predictive Modeling with Electronic Health Records [71.19967863320647]
  utilizing EHR data for predictive modeling presents several challenges due to its unique characteristics. Deep learning has demonstrated its superiority in various applications, including healthcare.
  arXiv  Detail & Related papers  (2024-02-02T00:31:01Z)
• Neuro-GPT: Towards A Foundation Model for EEG [0.04188114563181615]
  We propose Neuro-GPT, a foundation model consisting of an EEG encoder and a GPT model. Foundation model is pre-trained on a large-scale data set using a self-supervised task that learns how to reconstruct masked EEG segments. Experiments demonstrate that applying a foundation model can significantly improve classification performance compared to a model trained from scratch.
  arXiv  Detail & Related papers  (2023-11-07T07:07:18Z)
• Conditional Generative Models for Simulation of EMG During Naturalistic Movements [45.698312905115955]
  We present a conditional generative neural network trained adversarially to generate motor unit activation potential waveforms. We demonstrate the ability of such a model to predictively interpolate between a much smaller number of numerical model's outputs with a high accuracy.
  arXiv  Detail & Related papers  (2022-11-03T14:49:02Z)
• Mixed Effects Neural ODE: A Variational Approximation for Analyzing the Dynamics of Panel Data [50.23363975709122]
  We propose a probabilistic model called ME-NODE to incorporate (fixed + random) mixed effects for analyzing panel data. We show that our model can be derived using smooth approximations of SDEs provided by the Wong-Zakai theorem. We then derive Evidence Based Lower Bounds for ME-NODE, and develop (efficient) training algorithms.
  arXiv  Detail & Related papers  (2022-02-18T22:41:51Z)
• A multi-stage machine learning model on diagnosis of esophageal manometry [50.591267188664666]
  The framework includes deep-learning models at the swallow-level stage and feature-based machine learning models at the study-level stage. This is the first artificial-intelligence-style model to automatically predict CC diagnosis of HRM study from raw multi-swallow data.
  arXiv  Detail & Related papers  (2021-06-25T20:09:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.