Pretraining on Sleep Data Improves non-Sleep Biosignal Tasks

• URL: http://arxiv.org/abs/2605.02500v1
• Date: Mon, 04 May 2026 11:51:54 GMT
• Title: Pretraining on Sleep Data Improves non-Sleep Biosignal Tasks
• Abstract summary: Sleep foundation models have recently demonstrated strong performance on in-domain polysomnography tasks.<n>We investigate whether sleep biosignals can serve as an effective pretraining distribution for learning representations that transfer beyond sleep.
• Score: 5.729701207077404
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Sleep foundation models have recently demonstrated strong performance on in-domain polysomnography tasks, including sleep staging, apnea detection, and disease risk prediction. In this work, we investigate whether sleep biosignals can serve as an effective pretraining distribution for learning representations that transfer beyond sleep to adjacent domains. Following sleep foundation models, we perform sleep-only multimodal contrastive pretraining (with a leave-one-out objective) and evaluate transfer to non-sleep EEG and ECG, two well-benchmarked biosignal modalities with heterogeneous datasets and clinically meaningful downstream tasks. Across eight downstream tasks spanning multiple EEG and ECG datasets, sleep pretraining consistently improves performance relative to training from scratch. Moreover, on several tasks, we achieve performance competitive with or surpassing prior specialized state-of-the-art and foundation models.

Related papers

• Next-Token Prediction Learns Generalisable Representations of Sleep Physiology [3.80147466733132]
  We develop Hypnos, a sleep foundation model trained using eight different sensing modalities.<n>We tokenize each modality into streams of discrete tokens using residual vector quantization.<n>We train a large auto-regressive RQ-Transformer to jointly predict the next token across all modalities in parallel.
  arXiv  Detail & Related papers  (2026-06-08T15:13:57Z)
• Stanford Sleep Bench: Evaluating Polysomnography Pre-training Methods for Sleep Foundation Models [18.499236999143474]
  We release Stanford Sleep Bench, a large-scale PSG dataset comprising 17,467 recordings totaling over 163,000 hours from a major sleep clinic.<n>Our results show that multiple pretraining methods achieve comparable performance for sleep staging, apnea diagnosis, and age estimation.<n>To facilitate and advance sleep research, we will release Stanford Sleep Bench along with pretrained model weights, training pipelines, and evaluation code.
  arXiv  Detail & Related papers  (2025-12-10T12:37:29Z)
• impuTMAE: Multi-modal Transformer with Masked Pre-training for Missing Modalities Imputation in Cancer Survival Prediction [75.43342771863837]
  We introduce impuTMAE, a novel transformer-based end-to-end approach with an efficient multimodal pre-training strategy.<n>It learns inter- and intra-modal interactions while simultaneously imputing missing modalities by reconstructing masked patches.<n>Our model is pre-trained on heterogeneous, incomplete data and fine-tuned for glioma survival prediction using TCGA-GBM/LGG and BraTS datasets.
  arXiv  Detail & Related papers  (2025-08-08T10:01:16Z)
• Sleep Stage Classification using Multimodal Embedding Fusion from EOG and PSM [0.06282171844772422]
  This study introduces a novel approach that leverages ImageBind, a multimodal embedding deep learning model, to integrate PSM data with dual-channel EOG signals for sleep stage classification.<n>Our results demonstrate that fine-tuning ImageBind significantly improves classification accuracy, outperforming existing models.
  arXiv  Detail & Related papers  (2025-06-07T20:18:45Z)
• MetaSTH-Sleep: Towards Effective Few-Shot Sleep Stage Classification for Health Management with Spatial-Temporal Hypergraph Enhanced Meta-Learning [27.155280206930055]
  We propose MetaSTH-Sleep, a few-shot sleep stage classification framework based on spatial-temporal hypergraph enhanced meta-learning.<n>Our approach enables rapid adaptation to new subjects using only a few labeled samples, while the hypergraph structure effectively models complex spatial interconnections and temporal dynamics simultaneously in EEG signals.
  arXiv  Detail & Related papers  (2025-05-22T07:09:03Z)
• PSDNorm: Test-Time Temporal Normalization for Deep Learning in Sleep Staging [63.05435596565677]
  We propose PSDNorm that leverages Monge mapping and temporal context to normalize feature maps in deep learning models for signals.<n> PSDNorm achieves state-of-the-art performance on unseen left-out datasets while being 4-times more data-efficient than BatchNorm.
  arXiv  Detail & Related papers  (2025-03-06T16:20:25Z)
• Toward Foundational Model for Sleep Analysis Using a Multimodal Hybrid Self-Supervised Learning Framework [3.135883872525168]
  This study introduces SynthSleepNet, a multimodal hybrid self-supervised learning framework for analyzing polysomnography (PSG) data.<n> SynthSleepNet effectively integrates masked prediction and contrastive learning to leverage complementary features across multiple modalities.<n>It achieved superior performance compared to state-of-the-art methods across three downstream tasks.
  arXiv  Detail & Related papers  (2025-02-18T10:11:50Z)
• Day-Night Adaptation: An Innovative Source-free Adaptation Framework for Medical Image Segmentation [51.520294290813865]
  We propose a novel adaptation framework called Day-Night Adaptation (DyNA) with insights.<n>During the day, a low-frequency prompt is trained to adapt the frozen model to each test sample.<n>During the night, we reuse test data collected from the day and introduce a global student model to bridge the knowledge between teacher and student models.
  arXiv  Detail & Related papers  (2024-10-17T12:02:29Z)
• Convolutional Monge Mapping Normalization for learning on sleep data [63.22081662149488]
  We propose a new method called Convolutional Monge Mapping Normalization (CMMN) CMMN consists in filtering the signals in order to adapt their power spectrum density (PSD) to a Wasserstein barycenter estimated on training data. Numerical experiments on sleep EEG data show that CMMN leads to significant and consistent performance gains independent from the neural network architecture.
  arXiv  Detail & Related papers  (2023-05-30T08:24:01Z)
• On the Trade-off of Intra-/Inter-class Diversity for Supervised Pre-training [72.8087629914444]
  We study the impact of the trade-off between the intra-class diversity (the number of samples per class) and the inter-class diversity (the number of classes) of a supervised pre-training dataset. With the size of the pre-training dataset fixed, the best downstream performance comes with a balance on the intra-/inter-class diversity.
  arXiv  Detail & Related papers  (2023-05-20T16:23:50Z)
• Sleep Activity Recognition and Characterization from Multi-Source Passively Sensed Data [67.60224656603823]
  Sleep Activity Recognition methods can provide indicators to assess, monitor, and characterize subjects' sleep-wake cycles and detect behavioral changes. We propose a general method that continuously operates on passively sensed data from smartphones to characterize sleep and identify significant sleep episodes. Thanks to their ubiquity, these devices constitute an excellent alternative data source to profile subjects' biorhythms in a continuous, objective, and non-invasive manner.
  arXiv  Detail & Related papers  (2023-01-17T15:18:45Z)
• Convolutional Neural Networks for Sleep Stage Scoring on a Two-Channel EEG Signal [63.18666008322476]
  Sleep problems are one of the major diseases all over the world. Basic tool used by specialists is the Polysomnogram, which is a collection of different signals recorded during sleep. Specialists have to score the different signals according to one of the standard guidelines.
  arXiv  Detail & Related papers  (2021-03-30T09:59:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.