Multi-Channel Differential Transformer for Cross-Domain Sleep Stage Classification with Heterogeneous EEG and EOG

• URL: http://arxiv.org/abs/2508.15215v3
• Date: Fri, 26 Sep 2025 07:39:36 GMT
• Title: Multi-Channel Differential Transformer for Cross-Domain Sleep Stage Classification with Heterogeneous EEG and EOG
• Authors: Benjamin Wei Hao Chin, Yuin Torng Yew, Haocheng Wu, Lanxin Liang, Chow Khuen Chan, Norita Mohd Zain, Siti Balqis Samdin, Sim Kuan Goh,
• Abstract summary: We propose SleepDIFFormer, a multi-channel differential transformer framework for heterogeneous EEG-EOG representation learning.<n>SleepDIFFormer is trained across multiple sleep staging datasets, each treated as a source domain, with the goal of generalizing to unseen target domains.<n>Our approach mitigates spatial and temporal attention noise and learns a domain-invariant EEG-EOG representation through feature distribution alignment across datasets, thereby enhancing generalization to new domains.
• Score: 1.422074091840573
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Classification of sleep stages is essential for assessing sleep quality and diagnosing sleep disorders. However, manual inspection of EEG characteristics for each stage is time-consuming and prone to human error. Although machine learning and deep learning methods have been actively developed, they continue to face challenges arising from the non-stationarity and variability of electroencephalography (EEG) and electrooculography (EOG) signals across diverse clinical configurations, often resulting in poor generalization. In this work, we propose SleepDIFFormer, a multi-channel differential transformer framework for heterogeneous EEG-EOG representation learning. SleepDIFFormer is trained across multiple sleep staging datasets, each treated as a source domain, with the goal of generalizing to unseen target domains. Specifically, it employs a Multi-channel Differential Transformer Architecture (MDTA) designed to process raw EEG and EOG signals while incorporating cross-domain alignment. Our approach mitigates spatial and temporal attention noise and learns a domain-invariant EEG-EOG representation through feature distribution alignment across datasets, thereby enhancing generalization to new domains. Empirically, we evaluated SleepDIFFormer on five diverse sleep staging datasets under domain generalization settings and benchmarked it against existing approaches, achieving state-of-the-art performance. We further conducted a comprehensive ablation study and interpreted the differential attention weights, demonstrating their relevance to characteristic sleep EEG patterns. These findings advance the development of automated sleep stage classification and highlight its potential in quantifying sleep architecture and detecting abnormalities that disrupt restorative rest. Our source code and checkpoint are made publicly available at https://github.com/Ben1001409/SleepDIFFormer

Related papers

• NeurIPT: Foundation Model for Neural Interfaces [13.659630322026095]
  NeurIPT is a foundation model developed for diverse EEG-based Neural Interfaces with a Pre-trained Transformer.<n>We introduce Amplitude-Aware Masked Pretraining (AAMP) to learn robust representations across varying signal intensities beyond local.<n>We also develop Intra-Inter Lobe Pooling (IILP) during fine-tuning to efficiently exploit regional brain features.
  arXiv  Detail & Related papers  (2025-10-18T15:45:00Z)
• CodeBrain: Towards Decoupled Interpretability and Multi-Scale Architecture for EEG Foundation Model [52.466542039411515]
  EEG foundation models (EFMs) have emerged to address the scalability issues of task-specific models.<n>We present CodeBrain, a two-stage EFM designed to fill this gap.<n>In the first stage, we introduce the TFDual-Tokenizer, which decouples heterogeneous temporal and frequency EEG signals into discrete tokens.<n>In the second stage, we propose the multi-scale EEGSSM architecture, which combines structured global convolution with sliding window attention.
  arXiv  Detail & Related papers  (2025-06-10T17:20:39Z)
• Sleep Stage Classification using Multimodal Embedding Fusion from EOG and PSM [0.06282171844772422]
  This study introduces a novel approach that leverages ImageBind, a multimodal embedding deep learning model, to integrate PSM data with dual-channel EOG signals for sleep stage classification.<n>Our results demonstrate that fine-tuning ImageBind significantly improves classification accuracy, outperforming existing models.
  arXiv  Detail & Related papers  (2025-06-07T20:18:45Z)
• PSDNorm: Test-Time Temporal Normalization for Deep Learning in Sleep Staging [63.05435596565677]
  We propose PSDNorm that leverages Monge mapping and temporal context to normalize feature maps in deep learning models for signals.<n> PSDNorm achieves state-of-the-art performance on unseen left-out datasets while being 4-times more data-efficient than BatchNorm.
  arXiv  Detail & Related papers  (2025-03-06T16:20:25Z)
• LG-Sleep: Local and Global Temporal Dependencies for Mice Sleep Scoring [0.3141085922386211]
  This study introduces LG-Sleep, a novel subject-independent deep neural network architecture designed for mice sleep scoring through electroencephalogram (EEG) signals.<n>LG-Sleep extracts local and global temporal transitions within EEG signals to categorize sleep data into three stages: wake, rapid eye movement (REM) sleep, and non-rapid eye movement (NREM) sleep.
  arXiv  Detail & Related papers  (2024-12-19T21:39:57Z)
• Automatic Classification of Sleep Stages from EEG Signals Using Riemannian Metrics and Transformer Networks [6.404789669795639]
  In sleep medicine, assessing the evolution of a subject's sleep often involves the costly manual scoring of electroencephalographic (EEG) signals. We present a novel way of integrating learned signal-wise features into said matrices without sacrificing their Symmetric Definite Positive (SPD) nature.
  arXiv  Detail & Related papers  (2024-10-18T06:49:52Z)
• Multi-Source and Test-Time Domain Adaptation on Multivariate Signals using Spatio-Temporal Monge Alignment [59.75420353684495]
  Machine learning applications on signals such as computer vision or biomedical data often face challenges due to the variability that exists across hardware devices or session recordings. In this work, we propose Spatio-Temporal Monge Alignment (STMA) to mitigate these variabilities. We show that STMA leads to significant and consistent performance gains between datasets acquired with very different settings.
  arXiv  Detail & Related papers  (2024-07-19T13:33:38Z)
• Geodesic Optimization for Predictive Shift Adaptation on EEG data [53.58711912565724]
  Domain adaptation methods struggle when distribution shifts occur simultaneously in $X$ and $y$. This paper proposes a novel method termed Geodesic Optimization for Predictive Shift Adaptation (GOPSA) to address test-time multi-source DA. GOPSA has the potential to combine the advantages of mixed-effects modeling with machine learning for biomedical applications of EEG.
  arXiv  Detail & Related papers  (2024-07-04T12:15:42Z)
• DACAD: Domain Adaptation Contrastive Learning for Anomaly Detection in Multivariate Time Series [83.76994646443498]
  In time series anomaly detection, the scarcity of labeled data poses a challenge to the development of accurate models.<n>We propose a novel Domain Contrastive learning model for Anomaly Detection in time series (DACAD)<n>Our model employs supervised contrastive loss for the source domain and self-supervised contrastive triplet loss for the target domain.
  arXiv  Detail & Related papers  (2024-04-17T11:20:14Z)
• Multi-Source Domain Adaptation with Transformer-based Feature Generation for Subject-Independent EEG-based Emotion Recognition [0.5439020425819]
  We propose a multi-source domain adaptation approach with a transformer-based feature generator (MSDA-TF) designed to leverage information from multiple sources. During the adaptation process, we group the source subjects based on correlation values and aim to align the moments of the target subject with each source as well as within the sources. MSDA-TF is validated on the SEED dataset and is shown to yield promising results.
  arXiv  Detail & Related papers  (2024-01-04T16:38:47Z)
• Video Anomaly Detection via Spatio-Temporal Pseudo-Anomaly Generation : A Unified Approach [49.995833831087175]
  This work proposes a novel method for generating generic Video-temporal PAs by inpainting a masked out region of an image. In addition, we present a simple unified framework to detect real-world anomalies under the OCC setting. Our method performs on par with other existing state-of-the-art PAs generation and reconstruction based methods under the OCC setting.
  arXiv  Detail & Related papers  (2023-11-27T13:14:06Z)
• Enhancing Healthcare with EOG: A Novel Approach to Sleep Stage Classification [1.565361244756411]
  We introduce an innovative approach to automated sleep stage classification using EOG signals, addressing the discomfort and impracticality associated with EEG data acquisition. Our proposed SE-Resnet-Transformer model provides an accurate classification of five distinct sleep stages from raw EOG signal.
  arXiv  Detail & Related papers  (2023-09-25T16:23:39Z)
• Classification of sleep stages from EEG, EOG and EMG signals by SSNet [2.1915057426589746]
  Classification of sleep stages plays an essential role in diagnosing sleep-related diseases including Sleep Disorder Breathing (SDB) disease. We propose an end-to-end deep learning architecture, named SSNet, which comprises of two deep learning networks based on CNN andLSTM. Our model achieves the best performance in classifying sleep stages when compared with the state-of-the-art techniques.
  arXiv  Detail & Related papers  (2023-07-03T01:05:24Z)
• Convolutional Monge Mapping Normalization for learning on sleep data [63.22081662149488]
  We propose a new method called Convolutional Monge Mapping Normalization (CMMN) CMMN consists in filtering the signals in order to adapt their power spectrum density (PSD) to a Wasserstein barycenter estimated on training data. Numerical experiments on sleep EEG data show that CMMN leads to significant and consistent performance gains independent from the neural network architecture.
  arXiv  Detail & Related papers  (2023-05-30T08:24:01Z)
• Sleep Activity Recognition and Characterization from Multi-Source Passively Sensed Data [67.60224656603823]
  Sleep Activity Recognition methods can provide indicators to assess, monitor, and characterize subjects' sleep-wake cycles and detect behavioral changes. We propose a general method that continuously operates on passively sensed data from smartphones to characterize sleep and identify significant sleep episodes. Thanks to their ubiquity, these devices constitute an excellent alternative data source to profile subjects' biorhythms in a continuous, objective, and non-invasive manner.
  arXiv  Detail & Related papers  (2023-01-17T15:18:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.