Siamese Sleep Transformer For Robust Sleep Stage Scoring With Self-knowledge Distillation and Selective Batch Sampling

• URL: http://arxiv.org/abs/2212.13919v1
• Date: Mon, 12 Dec 2022 02:54:14 GMT
• Title: Siamese Sleep Transformer For Robust Sleep Stage Scoring With Self-knowledge Distillation and Selective Batch Sampling
• Abstract summary: We propose a Siamese sleep transformer (SST) that effectively extracts features from single-channel raw electroencephalogram signals for robust sleep stage scoring. To alleviate these problems, we propose the SST, a novel sleep stage scoring model with a selective batch sampling strategy and self-knowledge distillation.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this paper, we propose a Siamese sleep transformer (SST) that effectively extracts features from single-channel raw electroencephalogram signals for robust sleep stage scoring. Despite the significant advances in sleep stage scoring in the last few years, most of them mainly focused on the increment of model performance. However, other problems still exist: the bias of labels in datasets and the instability of model performance by repetitive training. To alleviate these problems, we propose the SST, a novel sleep stage scoring model with a selective batch sampling strategy and self-knowledge distillation. To evaluate how robust the model was to the bias of labels, we used different datasets for training and testing: the sleep heart health study and the Sleep-EDF datasets. In this condition, the SST showed competitive performance in sleep stage scoring. In addition, we demonstrated the effectiveness of the selective batch sampling strategy with a reduction of the standard deviation of performance by repetitive training. These results could show that SST extracted effective learning features against the bias of labels in datasets, and the selective batch sampling strategy worked for the model robustness in training.

Related papers

• Sleep-stage efficient classification using a lightweight self-supervised model [9.983526161001995]
  The mulEEG model, which learns electroencephalogram signal representations in a self-supervised manner, was simplified here.<n>The results showed that reducing the volume of data offered a better cost-benefit ratio compared to simplifying the model.
  arXiv  Detail & Related papers  (2026-05-25T19:37:54Z)
• Pretraining on Sleep Data Improves non-Sleep Biosignal Tasks [5.729701207077404]
  Sleep foundation models have recently demonstrated strong performance on in-domain polysomnography tasks.<n>We investigate whether sleep biosignals can serve as an effective pretraining distribution for learning representations that transfer beyond sleep.
  arXiv  Detail & Related papers  (2026-05-04T11:51:54Z)
• EEG Sleep Stage Classification with Continuous Wavelet Transform and Deep Learning [0.0]
  This study proposes a novel framework for automated sleep stage scoring using time-frequency analysis based on the wavelet transform.<n> Experimental results demonstrate that the proposed wavelet-based representation achieves an overall accuracy of 88.37 percent and a macro-averaged F1 score of 73.15.<n>These findings highlight the potential of wavelet analysis for robust, interpretable, and clinically applicable sleep stage classification.
  arXiv  Detail & Related papers  (2025-10-08T20:37:20Z)
• Enhancing Sample Selection by Cutting Mislabeled Easy Examples [62.13094877228772]
  We show that mislabeled examples correctly predicted by the model early in the training process are particularly harmful to model performance. We propose Early Cutting, which employs the model's later training state to re-select the confident subset identified early in training.
  arXiv  Detail & Related papers  (2025-02-12T09:12:45Z)
• Foster Adaptivity and Balance in Learning with Noisy Labels [26.309508654960354]
  We propose a novel approach named textbfSED to deal with label noise in a textbfSelf-adaptivtextbfE and class-balancetextbfD manner. A mean-teacher model is then employed to correct labels of noisy samples. We additionally propose a self-adaptive and class-balanced sample re-weighting mechanism to assign different weights to detected noisy samples.
  arXiv  Detail & Related papers  (2024-07-03T03:10:24Z)
• Learning with Imbalanced Noisy Data by Preventing Bias in Sample Selection [82.43311784594384]
  Real-world datasets contain not only noisy labels but also class imbalance. We propose a simple yet effective method to address noisy labels in imbalanced datasets.
  arXiv  Detail & Related papers  (2024-02-17T10:34:53Z)
• Noisy Correspondence Learning with Self-Reinforcing Errors Mitigation [63.180725016463974]
  Cross-modal retrieval relies on well-matched large-scale datasets that are laborious in practice. We introduce a novel noisy correspondence learning framework, namely textbfSelf-textbfReinforcing textbfErrors textbfMitigation (SREM)
  arXiv  Detail & Related papers  (2023-12-27T09:03:43Z)
• Transparency in Sleep Staging: Deep Learning Method for EEG Sleep Stage Classification with Model Interpretability [5.747465732334616]
  This study presents an end-to-end deep learning (DL) model which integrates squeeze and excitation blocks within the residual network to extract features and stacked Bi-LSTM to understand complex temporal dependencies. A distinctive aspect of this study is the adaptation of GradCam for sleep staging, marking the first instance of an explainable DL model in this domain with alignment of its decision-making with sleep expert's insights.
  arXiv  Detail & Related papers  (2023-09-10T17:56:03Z)
• SleepEGAN: A GAN-enhanced Ensemble Deep Learning Model for Imbalanced Classification of Sleep Stages [4.649202082648198]
  This paper develops a generative adversarial network (GAN)-powered ensemble deep learning model, named SleepEGAN, for the imbalanced classification of sleep stages. We show that the proposed method can improve classification accuracy compared to several existing state-of-the-art methods using three public sleep datasets.
  arXiv  Detail & Related papers  (2023-07-04T01:56:00Z)
• Selecting Learnable Training Samples is All DETRs Need in Crowded Pedestrian Detection [72.97320260601347]
  In crowded pedestrian detection, the performance of DETRs is still unsatisfactory due to the inappropriate sample selection method. We propose Sample Selection for Crowded Pedestrians, which consists of the constraint-guided label assignment scheme (CGLA) Experimental results show that the proposed SSCP effectively improves the baselines without introducing any overhead in inference.
  arXiv  Detail & Related papers  (2023-05-18T08:28:01Z)
• Temporal Output Discrepancy for Loss Estimation-based Active Learning [65.93767110342502]
  We present a novel deep active learning approach that queries the oracle for data annotation when the unlabeled sample is believed to incorporate high loss. Our approach achieves superior performances than the state-of-the-art active learning methods on image classification and semantic segmentation tasks.
  arXiv  Detail & Related papers  (2022-12-20T19:29:37Z)
• TTAPS: Test-Time Adaption by Aligning Prototypes using Self-Supervision [70.05605071885914]
  We propose a novel modification of the self-supervised training algorithm SwAV that adds the ability to adapt to single test samples. We show the success of our method on the common benchmark dataset CIFAR10-C.
  arXiv  Detail & Related papers  (2022-05-18T05:43:06Z)
• Efficient Test-Time Model Adaptation without Forgetting [60.36499845014649]
  Test-time adaptation seeks to tackle potential distribution shifts between training and testing data. We propose an active sample selection criterion to identify reliable and non-redundant samples. We also introduce a Fisher regularizer to constrain important model parameters from drastic changes.
  arXiv  Detail & Related papers  (2022-04-06T06:39:40Z)
• SleepTransformer: Automatic Sleep Staging with Interpretability and Uncertainty Quantification [19.51641720686276]
  Black-box skepticism is one of the main hindrances impeding deep-learning-based automatic sleep scoring from being used in clinical environments. This work proposes a sequence-to-sequence sleep-staging model, namely SleepTransformer. It is based on the transformer backbone whose self-attention scores offer interpretability of the model's decisions at both the epoch and sequence level.
  arXiv  Detail & Related papers  (2021-05-23T23:30:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.