SleepTransformer: Automatic Sleep Staging with Interpretability and Uncertainty Quantification

• URL: http://arxiv.org/abs/2105.11043v1
• Date: Sun, 23 May 2021 23:30:58 GMT
• Title: SleepTransformer: Automatic Sleep Staging with Interpretability and Uncertainty Quantification
• Abstract summary: Black-box skepticism is one of the main hindrances impeding deep-learning-based automatic sleep scoring from being used in clinical environments. This work proposes a sequence-to-sequence sleep-staging model, namely SleepTransformer. It is based on the transformer backbone whose self-attention scores offer interpretability of the model's decisions at both the epoch and sequence level.
• Score: 19.51641720686276
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Black-box skepticism is one of the main hindrances impeding deep-learning-based automatic sleep scoring from being used in clinical environments. Towards interpretability, this work proposes a sequence-to-sequence sleep-staging model, namely SleepTransformer. It is based on the transformer backbone whose self-attention scores offer interpretability of the model's decisions at both the epoch and sequence level. At the epoch level, the attention scores can be encoded as a heat map to highlight sleep-relevant features captured from the input EEG signal. At the sequence level, the attention scores are visualized as the influence of different neighboring epochs in an input sequence (i.e. the context) to recognition of a target epoch, mimicking the way manual scoring is done by human experts. We further propose a simple yet efficient method to quantify uncertainty in the model's decisions. The method, which is based on entropy, can serve as a metric for deferring low-confidence epochs to a human expert for further inspection. Additionally, we demonstrate that the proposed SleepTransformer outperforms existing methods at a lower computational cost and achieves state-of-the-art performance on two experimental databases of different sizes.

Related papers

• Rethinking Random Transformers as Adaptive Sequence Smoothers for Sleep Staging [17.920922817772624]
  We show that a randomly Kernel Transformer, without any training, substantially improves sleep staging performance.<n>We formalize this effect via a Random Attention Prior (RAPK), showing that random self-attention acts as an adaptive smoother.
  arXiv  Detail & Related papers  (2026-05-11T02:48:06Z)
• Self-Supervised Learning via Flow-Guided Neural Operator on Time-Series Data [57.85958428020496]
  Flow-Guided Neural Operator (FGNO) is a novel framework combining operator learning with flow matching for SSL training.<n>FGNO learns mappings in functional spaces by using Short-Time Fourier Transform to unify different time resolutions.<n>Unlike prior generative SSL methods that use noisy inputs during inference, we propose using clean inputs for representation extraction while learning representations with noise.
  arXiv  Detail & Related papers  (2026-02-12T18:54:57Z)
• DoPE: Denoising Rotary Position Embedding [60.779039511252584]
  Rotary Position Embedding (RoPE) in Transformer models has inherent limits that weaken length.<n>We reinterpret the attention map with positional encoding as a noisy feature map, and propose Denoising Positional extrapolation page (DoPE)<n>DoPE is a training-free method based on truncated matrix entropy to detect outlier frequency bands in the feature map.
  arXiv  Detail & Related papers  (2025-11-12T09:32:35Z)
• EEG Sleep Stage Classification with Continuous Wavelet Transform and Deep Learning [0.0]
  This study proposes a novel framework for automated sleep stage scoring using time-frequency analysis based on the wavelet transform.<n> Experimental results demonstrate that the proposed wavelet-based representation achieves an overall accuracy of 88.37 percent and a macro-averaged F1 score of 73.15.<n>These findings highlight the potential of wavelet analysis for robust, interpretable, and clinically applicable sleep stage classification.
  arXiv  Detail & Related papers  (2025-10-08T20:37:20Z)
• Investigating Stochastic Methods for Prosody Modeling in Speech Synthesis [59.68341193176134]
  generative methods have progressed rapidly in recent years, but generating expressive prosody for an utterance remains a challenging task.<n>In this work, we investigate the effectiveness of methods for this task, including Normalizing Flows, Conditional Flow Matching, and Rectified Flows.<n>Our extensive subjective and objective evaluations demonstrate that methods produce natural prosody on par with human speakers by capturing the inherent variability in human speech.
  arXiv  Detail & Related papers  (2025-06-30T19:52:32Z)
• LG-Sleep: Local and Global Temporal Dependencies for Mice Sleep Scoring [0.3141085922386211]
  This study introduces LG-Sleep, a novel subject-independent deep neural network architecture designed for mice sleep scoring through electroencephalogram (EEG) signals. LG-Sleep extracts local and global temporal transitions within EEG signals to categorize sleep data into three stages: wake, rapid eye movement (REM) sleep, and non-rapid eye movement (NREM) sleep.
  arXiv  Detail & Related papers  (2024-12-19T21:39:57Z)
• Automatic Classification of Sleep Stages from EEG Signals Using Riemannian Metrics and Transformer Networks [6.404789669795639]
  In sleep medicine, assessing the evolution of a subject's sleep often involves the costly manual scoring of electroencephalographic (EEG) signals. We present a novel way of integrating learned signal-wise features into said matrices without sacrificing their Symmetric Definite Positive (SPD) nature.
  arXiv  Detail & Related papers  (2024-10-18T06:49:52Z)
• DAPE V2: Process Attention Score as Feature Map for Length Extrapolation [63.87956583202729]
  We conceptualize attention as a feature map and apply the convolution operator to mimic the processing methods in computer vision. The novel insight, which can be adapted to various attention-related models, reveals that the current Transformer architecture has the potential for further evolution.
  arXiv  Detail & Related papers  (2024-10-07T07:21:49Z)
• PseudoNeg-MAE: Self-Supervised Point Cloud Learning using Conditional Pseudo-Negative Embeddings [55.55445978692678]
  PseudoNeg-MAE is a self-supervised learning framework that enhances global feature representation of point cloud mask autoencoders. We show that PseudoNeg-MAE achieves state-of-the-art performance on the ModelNet40 and ScanObjectNN datasets.
  arXiv  Detail & Related papers  (2024-09-24T07:57:21Z)
• Time-series Generation by Contrastive Imitation [87.51882102248395]
  We study a generative framework that seeks to combine the strengths of both: Motivated by a moment-matching objective to mitigate compounding error, we optimize a local (but forward-looking) transition policy. At inference, the learned policy serves as the generator for iterative sampling, and the learned energy serves as a trajectory-level measure for evaluating sample quality.
  arXiv  Detail & Related papers  (2023-11-02T16:45:25Z)
• Sleep Activity Recognition and Characterization from Multi-Source Passively Sensed Data [67.60224656603823]
  Sleep Activity Recognition methods can provide indicators to assess, monitor, and characterize subjects' sleep-wake cycles and detect behavioral changes. We propose a general method that continuously operates on passively sensed data from smartphones to characterize sleep and identify significant sleep episodes. Thanks to their ubiquity, these devices constitute an excellent alternative data source to profile subjects' biorhythms in a continuous, objective, and non-invasive manner.
  arXiv  Detail & Related papers  (2023-01-17T15:18:45Z)
• Siamese Sleep Transformer For Robust Sleep Stage Scoring With Self-knowledge Distillation and Selective Batch Sampling [0.0]
  We propose a Siamese sleep transformer (SST) that effectively extracts features from single-channel raw electroencephalogram signals for robust sleep stage scoring. To alleviate these problems, we propose the SST, a novel sleep stage scoring model with a selective batch sampling strategy and self-knowledge distillation.
  arXiv  Detail & Related papers  (2022-12-12T02:54:14Z)
• SleePyCo: Automatic Sleep Scoring with Feature Pyramid and Contrastive Learning [0.0]
  We propose a deep learning framework named SleePyCo that incorporates 1) a feature pyramid and 2) supervised contrastive learning for automatic sleep scoring. For the feature pyramid, we propose a backbone network named SleePyCo-backbone to consider multiple feature sequences on different temporal and frequency scales. Supervised contrastive learning allows the network to extract class discriminative features by minimizing the distance between intra-class features and simultaneously maximizing that between inter-class features.
  arXiv  Detail & Related papers  (2022-09-20T04:10:49Z)
• Adaptive Spike-Like Representation of EEG Signals for Sleep Stages Scoring [6.644008481573341]
  We propose an adaptive scheme to encode, filter and accumulate the input signals and the weight features by the half-Gaussian probabilities of signal intensities. Experiments on the largest public dataset against state-of-the-art methods validate the effectiveness of our proposed method and reveal promising future directions.
  arXiv  Detail & Related papers  (2022-04-02T11:21:49Z)
• TransSleep: Transitioning-aware Attention-based Deep Neural Network for Sleep Staging [2.105172041656126]
  We propose a novel deep neural network structure, TransSleep, that captures distinctive local temporal patterns. Results show that TransSleep achieves promising performance in automatic sleep staging.
  arXiv  Detail & Related papers  (2022-03-22T08:55:32Z)
• Transformers Can Do Bayesian Inference [56.99390658880008]
  We present Prior-Data Fitted Networks (PFNs) PFNs leverage in-context learning in large-scale machine learning techniques to approximate a large set of posteriors. We demonstrate that PFNs can near-perfectly mimic Gaussian processes and also enable efficient Bayesian inference for intractable problems.
  arXiv  Detail & Related papers  (2021-12-20T13:07:39Z)
• Autoencoding Variational Autoencoder [56.05008520271406]
  We study the implications of this behaviour on the learned representations and also the consequences of fixing it by introducing a notion of self consistency. We show that encoders trained with our self-consistency approach lead to representations that are robust (insensitive) to perturbations in the input introduced by adversarial attacks.
  arXiv  Detail & Related papers  (2020-12-07T14:16:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.