OmniEEG-Bench: A Standardized Evaluation Benchmark for EEG Foundation Models

• URL: http://arxiv.org/abs/2606.00815v1
• Date: Sat, 30 May 2026 17:20:04 GMT
• Title: OmniEEG-Bench: A Standardized Evaluation Benchmark for EEG Foundation Models
• Abstract summary: We introduce OmniEEG-Bench, a unified benchmark and downstream task roadmap for EEG foundation models (FMs)<n>It organizes evaluation of EEG FMs into six task families spanning (i) signal reliability, (ii) biometrics and disease, (iii) consciousness and state, (iv) cognition and emotion, (v) naturalistic stimulus decoding, and (vi) motor and interaction.<n>We benchmark 10 representative EEG foundation models and report a leaderboard that covers diverse evaluation settings.
• Score: 22.964421663748755
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Electroencephalography (EEG) supports a variety of brain-computer interface (BCI) tasks ranging from brain-state monitoring to human-LLM interactions. EEG foundation models are emerging, but evaluation remains fragmented due to heterogeneous datasets and nconsistent task protocols. Here, we introduce OmniEEG-Bench, a unified benchmark and downstream task roadmap for EEG foundation models (FMs). It organizes evaluation of EEG FMs into six task families spanning (i) signal reliability, (ii) biometrics and disease, (iii) consciousness and state, (iv) cognition and emotion, (v) naturalistic stimulus decoding, and (vi) motor and interaction, introducing a new generation of tasks not systematically benchmarked in prior EEG FM work. OmniEEG-Bench standardizes model deployment, task definitions, and metrics through a task-card specification, and unifies 54 EEG datasets with consistent evaluation protocols. We benchmark 10 representative EEG foundation models and report a leaderboard that covers diverse evaluation settings. Both pretraining dataset diversity and model size are significantly associated with better average ranks across datasets, revealing scaling-law behavior in EEG foundation models (Figure 1). These results suggest that scaling EEG foundation models requires not only larger architectures but also broader and more diverse pretraining data. The benchmark code is available at https://github.com/ncclab-sustech/omni-eegbench.git.

Related papers

• Taming foundation model with invariance-oriented pre-training for broad-spectrum EEG analysis across signal-level, brain-state, and brain-health tasks [46.533103231728724]
  INCEPT is an invariance-oriented EEG foundation model trained on over 11,000 hours of unlabelled clinical EEG.<n>We evaluate INCEPT on a broad-spectrum benchmark of ten datasets spanning three levels of post-acquisition EEG analysis.
  arXiv  Detail & Related papers  (2026-08-25T14:20:02Z)
• A Multi-dimensional Framework for Evaluating Generalization in EEG Foundation Models [48.385356148620815]
  Recent EEG foundation models have demonstrated promising transfer capabilities across tasks and datasets.<n>We propose a multi-dimensional evaluation framework for assessing EEG models under realistic low-resource conditions.
  arXiv  Detail & Related papers  (2026-05-27T14:51:54Z)
• GEO-Bench-2: From Performance to Capability, Rethinking Evaluation in Geospatial AI [52.13138825802668]
  GeoFMs are transforming Earth Observation, but evaluation lacks standardized protocols.<n> GEO-Bench-2 addresses this with a comprehensive framework spanning classification, segmentation, regression, object detection, and instance segmentation.<n>Code, data, and leaderboard for GEO-Bench-2 are publicly released under a permissive license.
  arXiv  Detail & Related papers  (2025-11-19T17:45:02Z)
• EEG-FM-Bench: A Comprehensive Benchmark for the Systematic Evaluation of EEG Foundation Models [16.433809341013113]
  EEG-FM-Bench is the first comprehensive benchmark for the systematic and standardized evaluation of EEG foundation models (EEG-FMs)<n>Our contributions are threefold: (1) we curate a diverse suite of downstream tasks and datasets from canonical EEG paradigms, implementing standardized processing and evaluation protocols within a unified open-source framework; (2) we benchmark prominent state-of-the-art foundation models to establish comprehensive baseline results for a clear comparison of the current landscape; (3) we perform qualitative analyses to provide insights into model behavior and inform future architectural design.
  arXiv  Detail & Related papers  (2025-08-25T07:34:33Z)
• EAD: An EEG Adapter for Automated Classification [6.587374692735942]
  EEG Adapter (EAD) is a flexible framework compatible with any signal acquisition device.<n>We leverage a recent EEG model with significant adaptations to learn robust representations from the EEG data for the classification task.<n>We evaluate EAD on two publicly available datasets achieving state-of-the-art accuracies 99.33% and 92.31% on EEG-ImageNet and BrainLat respectively.
  arXiv  Detail & Related papers  (2025-05-29T05:21:06Z)
• BrainOmni: A Brain Foundation Model for Unified EEG and MEG Signals [46.121056431476156]
  This paper proposes Brain Omni, the first brain foundation model that generalises across heterogeneous EEG and MEG recordings.<n>Existing approaches typically rely on separate, modality- and dataset-specific models, which limits performance and cross-domain scalability.<n>A total of 1,997 hours of EEG and 656 hours of MEG data are curated and standardised from publicly available sources for pretraining.
  arXiv  Detail & Related papers  (2025-05-18T14:07:14Z)
• GEFM: Graph-Enhanced EEG Foundation Model [16.335330142000657]
  Foundation models offer a promising solution by leveraging large-scale unlabeled data through pre-training.<n>We propose Graph-Enhanced EEG Foundation Model (GEFM), a novel foundation model for EEG that integrates both temporal and inter-channel information.<n>Our architecture combines Graph Neural Networks (GNNs), which effectively capture relational structures, with a masked autoencoder to enable efficient pre-training.
  arXiv  Detail & Related papers  (2024-11-29T06:57:50Z)
• BrainGPT: Unleashing the Potential of EEG Generalist Foundation Model by Autoregressive Pre-training [15.135177893151008]
  EEGPT is the first generalist EEG foundation model designed to address these challenges.<n>First, we propose an electrode-wise modeling strategy that treats each electrode as a fundamental unit.<n>Second, we develop the first autoregressive EEG pre-trained model.<n>Third, we introduce a multi-task transfer learning paradigm using a learnable electrode graph network.
  arXiv  Detail & Related papers  (2024-10-14T12:17:54Z)
• GenBench: A Benchmarking Suite for Systematic Evaluation of Genomic Foundation Models [56.63218531256961]
  We introduce GenBench, a benchmarking suite specifically tailored for evaluating the efficacy of Genomic Foundation Models. GenBench offers a modular and expandable framework that encapsulates a variety of state-of-the-art methodologies. We provide a nuanced analysis of the interplay between model architecture and dataset characteristics on task-specific performance.
  arXiv  Detail & Related papers  (2024-06-01T08:01:05Z)
• Neuro-GPT: Towards A Foundation Model for EEG [0.04188114563181615]
  We propose Neuro-GPT, a foundation model consisting of an EEG encoder and a GPT model. Foundation model is pre-trained on a large-scale data set using a self-supervised task that learns how to reconstruct masked EEG segments. Experiments demonstrate that applying a foundation model can significantly improve classification performance compared to a model trained from scratch.
  arXiv  Detail & Related papers  (2023-11-07T07:07:18Z)
• EEG-Inception: An Accurate and Robust End-to-End Neural Network for EEG-based Motor Imagery Classification [123.93460670568554]
  This paper proposes a novel convolutional neural network (CNN) architecture for accurate and robust EEG-based motor imagery (MI) classification. The proposed CNN model, namely EEG-Inception, is built on the backbone of the Inception-Time network. The proposed network is an end-to-end classification, as it takes the raw EEG signals as the input and does not require complex EEG signal-preprocessing.
  arXiv  Detail & Related papers  (2021-01-24T19:03:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.