ECHO: Toward Contextual Seq2Seq Paradigms in Large EEG Models

• URL: http://arxiv.org/abs/2509.22556v1
• Date: Fri, 26 Sep 2025 16:37:34 GMT
• Title: ECHO: Toward Contextual Seq2Seq Paradigms in Large EEG Models
• Authors: Chenyu Liu, Yuqiu Deng, Tianyu Liu, Jinan Zhou, Xinliang Zhou, Ziyu Jia, Yi Ding,
• Abstract summary: Large EEG Models (LEMs) address this by pretraining encoder-centric architectures on large-scale unlabeled data to extract universal representations.<n>We introduce ECHO, a novel decoder-centric LEM paradigm that reformulates EEG modeling as sequence-to-sequence learning.<n>ECHO consistently outperforms state-of-the-art single-task LEMs in multi-task settings, showing superior generalization and adaptability.
• Score: 28.18721116424753
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Electroencephalography (EEG), with its broad range of applications, necessitates models that can generalize effectively across various tasks and datasets. Large EEG Models (LEMs) address this by pretraining encoder-centric architectures on large-scale unlabeled data to extract universal representations. While effective, these models lack decoders of comparable capacity, limiting the full utilization of the learned features. To address this issue, we introduce ECHO, a novel decoder-centric LEM paradigm that reformulates EEG modeling as sequence-to-sequence learning. ECHO captures layered relationships among signals, labels, and tasks within sequence space, while incorporating discrete support samples to construct contextual cues. This design equips ECHO with in-context learning, enabling dynamic adaptation to heterogeneous tasks without parameter updates. Extensive experiments across multiple datasets demonstrate that, even with basic model components, ECHO consistently outperforms state-of-the-art single-task LEMs in multi-task settings, showing superior generalization and adaptability.

Related papers

• OFA-MAS: One-for-All Multi-Agent System Topology Design based on Mixture-of-Experts Graph Generative Models [57.94189874119267]
  Multi-Agent Systems (MAS) offer a powerful paradigm for solving complex problems.<n>Current graph learning-based design methodologies often adhere to a "one-for-one" paradigm.<n>We propose OFA-TAD, a one-for-all framework that generates adaptive collaboration graphs for any task described in natural language.
  arXiv  Detail & Related papers  (2026-01-19T12:23:44Z)
• AR-MOT: Autoregressive Multi-object Tracking [56.09738000988466]
  We propose a novel autoregressive paradigm that formulates MOT as a sequence generation task within a large language model (LLM) framework.<n>This design enables the model to output structured results through flexible sequence construction, without requiring any task-specific heads.<n>To enhance region-level visual perception, we introduce an Object Tokenizer based on a pretrained detector.
  arXiv  Detail & Related papers  (2026-01-05T09:17:28Z)
• MultiDiffNet: A Multi-Objective Diffusion Framework for Generalizable Brain Decoding [1.6528632644902828]
  We introduce textitMultiDiffNet, a diffusion-based framework that bypasses generative augmentation entirely by learning a compact latent space optimized for multiple objectives.<n>We decode directly from this space and achieve state-of-the-art generalization across various neural decoding tasks using subject and session disjoint evaluation.
  arXiv  Detail & Related papers  (2025-11-23T05:22:27Z)
• SITS-DECO: A Generative Decoder Is All You Need For Multitask Satellite Image Time Series Modelling [0.0]
  We introduce SITS-DECO, a proof-of-concept generative model that applies unified-sequence framing to EO data.<n>We show that the model can perform multiple supervised and self-supervised tasks within a single unified architecture.<n>Despite its simplicity and lack of spatial context, SITS-DECO outperforms much larger EO foundation models on crop-type classification.
  arXiv  Detail & Related papers  (2025-10-21T14:42:55Z)
• BERT4beam: Large AI Model Enabled Generalized Beamforming Optimization [77.17508487745026]
  This paper investigates the large-scale AI model designed for beamforming optimization to adapt and generalize to diverse tasks defined by system utilities and scales.<n>We propose a novel framework based on bidirectional encoder representations from transformers (BERT), termed BERT4 encoder.<n>Based on the framework, we propose two BERT-based approaches for single-task and multi-task beamforming optimization, respectively.
  arXiv  Detail & Related papers  (2025-09-14T02:49:29Z)
• CRIA: A Cross-View Interaction and Instance-Adapted Pre-training Framework for Generalizable EEG Representations [52.251569042852815]
  CRIA is an adaptive framework that utilizes variable-length and variable-channel coding to achieve a unified representation of EEG data across different datasets.<n>The model employs a cross-attention mechanism to fuse temporal, spectral, and spatial features effectively.<n> Experimental results on the Temple University EEG corpus and the CHB-MIT dataset show that CRIA outperforms existing methods with the same pre-training conditions.
  arXiv  Detail & Related papers  (2025-06-19T06:31:08Z)
• Spatial-Temporal-Spectral Unified Modeling for Remote Sensing Dense Prediction [20.1863553357121]
  Current deep learning architectures for remote sensing are fundamentally rigid.<n>We introduce the Spatial-Temporal-Spectral Unified Network (STSUN) for unified modeling.<n> STSUN can adapt to input and output data with arbitrary spatial sizes, temporal lengths, and spectral bands.<n>It unifies various dense prediction tasks and diverse semantic class predictions.
  arXiv  Detail & Related papers  (2025-05-18T07:39:17Z)
• Single Domain Generalization with Model-aware Parametric Batch-wise Mixup [22.709796153794507]
  Single Domain Generalization remains a formidable challenge in the field of machine learning.<n>We propose a novel data augmentation approach, named as Model-aware Parametric Batch-wise Mixup.<n>By exploiting inter-feature correlations, the parameterized mixup generator introduces additional versatility in combining features across a batch of instances.
  arXiv  Detail & Related papers  (2025-02-22T03:45:18Z)
• BrainGPT: Unleashing the Potential of EEG Generalist Foundation Model by Autoregressive Pre-training [15.135177893151008]
  EEGPT is the first generalist EEG foundation model designed to address these challenges.<n>First, we propose an electrode-wise modeling strategy that treats each electrode as a fundamental unit.<n>Second, we develop the first autoregressive EEG pre-trained model.<n>Third, we introduce a multi-task transfer learning paradigm using a learnable electrode graph network.
  arXiv  Detail & Related papers  (2024-10-14T12:17:54Z)
• EEGMamba: Bidirectional State Space Model with Mixture of Experts for EEG Multi-task Classification [1.4004287903552533]
  We introduce EEGMamba, the first universal EEG classification network to truly implement multi-task learning for EEG applications. EEGMamba seamlessly integrates the Spatio-Temporal-Adaptive (ST- adaptive) module, bidirectional Mamba, and Mixture of Experts (MoE) into a unified framework. We evaluate our model on eight publicly available EEG datasets, and the experimental results demonstrate its superior performance in four types of tasks.
  arXiv  Detail & Related papers  (2024-07-20T11:15:47Z)
• Unified Generation, Reconstruction, and Representation: Generalized Diffusion with Adaptive Latent Encoding-Decoding [90.77521413857448]
  Deep generative models are anchored in three core capabilities -- generating new instances, reconstructing inputs, and learning compact representations. We introduce Generalized generative adversarial-Decoding Diffusion Probabilistic Models (EDDPMs) EDDPMs generalize the Gaussian noising-denoising in standard diffusion by introducing parameterized encoding-decoding. Experiments on text, proteins, and images demonstrate the flexibility to handle diverse data and tasks.
  arXiv  Detail & Related papers  (2024-02-29T10:08:57Z)
• StableLLaVA: Enhanced Visual Instruction Tuning with Synthesized Image-Dialogue Data [129.92449761766025]
  We propose a novel data collection methodology that synchronously synthesizes images and dialogues for visual instruction tuning. This approach harnesses the power of generative models, marrying the abilities of ChatGPT and text-to-image generative models. Our research includes comprehensive experiments conducted on various datasets.
  arXiv  Detail & Related papers  (2023-08-20T12:43:52Z)
• BENDR: using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG data [15.71234837305808]
  We consider how to adapt techniques and architectures used for language modelling (LM) to encephalography modelling (EM) We find that a single pre-trained model is capable of modelling completely novel raw EEG sequences recorded with differing hardware. Both the internal representations of this model and the entire architecture can be fine-tuned to a variety of downstream BCI and EEG classification tasks.
  arXiv  Detail & Related papers  (2021-01-28T14:54:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.