CoCoT-EEG: Contrastive-Pretrained Multiscale Convolutional Transformer for EEG Decoding

• URL: http://arxiv.org/abs/2607.09543v1
• Date: Fri, 10 Jul 2026 15:48:15 GMT
• Title: CoCoT-EEG: Contrastive-Pretrained Multiscale Convolutional Transformer for EEG Decoding
• Abstract summary: We develop a novel contrastive-pretrained EEG model with multiscale temporal convolution input layers and Transformer encoder blocks (CoCoT)<n>CoCoT matches or beats state-of-the-art reconstruction-pretrained EEG models on extensive benchmark decoding tasks with heterogeneous electrode configurations.<n>We demonstrate the viability of contrastive learning for building EEG FMs while suggesting key architectural design considerations.
• Score: 5.179334032083694
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Self-supervised pretrained foundation models (FM) have shown early promise for non-invasive electroencephalogram (EEG) decoding applications. Many recent large-scale models converged on the approach of tokenizing raw EEG followed by masked reconstruction pretraining. However, this recipe has been shown to be suboptimal for data, like EEG, with high noise amplitude and information confined to limited dimensions such as narrow frequency bands. Building on this insight, we develop a novel contrastive-pretrained EEG model with multiscale temporal convolution input layers and Transformer encoder blocks (CoCoT). CoCoT matches or beats state-of-the-art reconstruction-pretrained EEG models on extensive benchmark decoding tasks with heterogeneous electrode configurations. Furthermore, CoCoT trained from scratch outperforms previous single-task decoding models and even rivals pretrained models, showcasing the architecture's flexibility and data efficiency. Through systematic ablations, including model architecture and pretraining objective, we demonstrate the viability of contrastive learning for building EEG FMs while suggesting key architectural design considerations, prompting further investigations in alternative large-scale pretraining strategies.

Related papers

• EEG-PRIME: Prototype-Aligned Representation Learning with Multi-Level Conditioning for EEG Decoding [26.89647605331463]
  EEG-PRIME is a two-stage EEG foundation model for cross-dataset multi-task decoding.<n>During instruction tuning, EEG-PRIME incorporates task-semantic, dataset-specific, and subject-invariant conditioning.<n>Experiments show consistent improvements over state-of-the-art baselines and prior EEG foundation models under cross-subject settings.
  arXiv  Detail & Related papers  (2026-08-13T10:35:51Z)
• STEAM: A Spatio-TEmporal Alignment Mixture-of-Experts Model with Hierarchical Pre-training for EEG Decoding [12.49365160717963]
  We present STEAM, a hierarchical transfer framework that reconciles general-purpose representation learning with paradigm-specific specialization in EEG foundation models.<n> STEAM attains the best average rank among the compared methods at a competitive inference cost measured in FLOPs.
  arXiv  Detail & Related papers  (2026-08-03T11:15:22Z)
• SpecMoE: Spectral Mixture-of-Experts Foundation Model for Cross-Species EEG Decoding [0.0]
  SpecMoE is a mixture of experts framework guided by a learned spectral gating mechanism.<n>It achieves state-of-the-art performance across a diverse set of EEG decoding tasks, including sleep staging, emotion recognition, motor imagery classification, abnormal signal detection, and drug effect prediction.
  arXiv  Detail & Related papers  (2026-03-17T16:20:14Z)
• REVE: A Foundation Model for EEG -- Adapting to Any Setup with Large-Scale Pretraining on 25,000 Subjects [5.368295573908306]
  REVE (Representation for EEG with Versatile Embeddings) is a pretrained model explicitly designed to generalize across diverse EEG signals.<n>We pretrain REVE on over 60,000 hours of EEG data from 92 datasets spanning 25,000 subjects, representing the largest EEG pretraining effort to date.<n>We release code, pretrained weights, and tutorials to support standardized EEG research and accelerate progress in clinical neuroscience.
  arXiv  Detail & Related papers  (2025-10-24T15:52:46Z)
• EEGDM: EEG Representation Learning via Generative Diffusion Model [17.595769291603688]
  We propose an EEG representation learning framework building upon Generative Diffusion Model (EEGDM)<n>Specifically, we developed a structured state-space model for diffusion pretraining and trained it using Denoising Diffusion Probabilistic Model (DDPM) framework.<n>The resulting latent EEG representations were then used for downstream classification tasks via our proposed latent fusion transformer (LFT)
  arXiv  Detail & Related papers  (2025-08-13T14:40:52Z)
• Generalized Factor Neural Network Model for High-dimensional Regression [50.554377879576066]
  We tackle the challenges of modeling high-dimensional data sets with latent low-dimensional structures hidden within complex, non-linear, and noisy relationships.<n>Our approach enables a seamless integration of concepts from non-parametric regression, factor models, and neural networks for high-dimensional regression.
  arXiv  Detail & Related papers  (2025-02-16T23:13:55Z)
• CognitionCapturer: Decoding Visual Stimuli From Human EEG Signal With Multimodal Information [61.1904164368732]
  We propose CognitionCapturer, a unified framework that fully leverages multimodal data to represent EEG signals.<n>Specifically, CognitionCapturer trains Modality Experts for each modality to extract cross-modal information from the EEG modality.<n>The framework does not require any fine-tuning of the generative models and can be extended to incorporate more modalities.
  arXiv  Detail & Related papers  (2024-12-13T16:27:54Z)
• Exploring the design space of deep-learning-based weather forecasting systems [56.129148006412855]
  This paper systematically analyzes the impact of different design choices on deep-learning-based weather forecasting systems. We study fixed-grid architectures such as UNet, fully convolutional architectures, and transformer-based models. We propose a hybrid system that combines the strong performance of fixed-grid models with the flexibility of grid-invariant architectures.
  arXiv  Detail & Related papers  (2024-10-09T22:25:50Z)
• Mechanistic Design and Scaling of Hybrid Architectures [114.3129802943915]
  We identify and test new hybrid architectures constructed from a variety of computational primitives. We experimentally validate the resulting architectures via an extensive compute-optimal and a new state-optimal scaling law analysis. We find MAD synthetics to correlate with compute-optimal perplexity, enabling accurate evaluation of new architectures.
  arXiv  Detail & Related papers  (2024-03-26T16:33:12Z)
• End-to-End Meta-Bayesian Optimisation with Transformer Neural Processes [52.818579746354665]
  This paper proposes the first end-to-end differentiable meta-BO framework that generalises neural processes to learn acquisition functions via transformer architectures. We enable this end-to-end framework with reinforcement learning (RL) to tackle the lack of labelled acquisition data.
  arXiv  Detail & Related papers  (2023-05-25T10:58:46Z)
• CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learning [92.36705236706678]
  "CodeRL" is a new framework for program synthesis tasks through pretrained LMs and deep reinforcement learning. During inference, we introduce a new generation procedure with a critical sampling strategy. For the model backbones, we extended the encoder-decoder architecture of CodeT5 with enhanced learning objectives.
  arXiv  Detail & Related papers  (2022-07-05T02:42:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.