SleepFM: Multi-modal Representation Learning for Sleep Across Brain Activity, ECG and Respiratory Signals

• URL: http://arxiv.org/abs/2405.17766v1
• Date: Tue, 28 May 2024 02:43:53 GMT
• Title: SleepFM: Multi-modal Representation Learning for Sleep Across Brain Activity, ECG and Respiratory Signals
• Authors: Rahul Thapa, Bryan He, Magnus Ruud Kjaer, Hyatt Moore, Gauri Ganjoo, Emmanuel Mignot, James Zou,
• Abstract summary: Sleep is a complex physiological process evaluated through various modalities recording electrical brain, cardiac, and respiratory activities. We developed SleepFM, the first multi-modal foundation model for sleep analysis. We show that a novel leave-one-out approach for contrastive learning significantly improves downstream task performance.
• Score: 17.416001617612658
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Sleep is a complex physiological process evaluated through various modalities recording electrical brain, cardiac, and respiratory activities. We curate a large polysomnography dataset from over 14,000 participants comprising over 100,000 hours of multi-modal sleep recordings. Leveraging this extensive dataset, we developed SleepFM, the first multi-modal foundation model for sleep analysis. We show that a novel leave-one-out approach for contrastive learning significantly improves downstream task performance compared to representations from standard pairwise contrastive learning. A logistic regression model trained on SleepFM's learned embeddings outperforms an end-to-end trained convolutional neural network (CNN) on sleep stage classification (macro AUROC 0.88 vs 0.72 and macro AUPRC 0.72 vs 0.48) and sleep disordered breathing detection (AUROC 0.85 vs 0.69 and AUPRC 0.77 vs 0.61). Notably, the learned embeddings achieve 48% top-1 average accuracy in retrieving the corresponding recording clips of other modalities from 90,000 candidates. This work demonstrates the value of holistic multi-modal sleep modeling to fully capture the richness of sleep recordings. SleepFM is open source and available at https://github.com/rthapa84/sleepfm-codebase.

Related papers

• ProductGraphSleepNet: Sleep Staging using Product Spatio-Temporal Graph Learning with Attentive Temporal Aggregation [4.014524824655106]
  This work proposes an adaptive product graph learning-based graph convolutional network, named ProductGraphSleepNet, for learning joint-temporal graphs. The proposed network makes it possible for clinicians to comprehend and interpret the learned connectivity graphs for sleep stages.
  arXiv  Detail & Related papers  (2022-12-09T14:34:58Z)
• Extraction of Sleep Information from Clinical Notes of Patients with Alzheimer's Disease Using Natural Language Processing [4.268772592648502]
  Sleep is one of the lifestyle-related factors that has been shown critical for optimal cognitive function in old age. Traditional way to acquire sleep information is time-consuming, inefficient, non-scalable, and limited to patients' subjective experience. We developed a rule-based Natural Language Processing (NLP) algorithm, machine learning models, and Large Language Model(LLM)-based NLP algorithms to automate the extraction of sleep-related concepts.
  arXiv  Detail & Related papers  (2022-03-08T21:20:19Z)
• Ensemble of Convolution Neural Networks on Heterogeneous Signals for Sleep Stage Scoring [63.30661835412352]
  This paper explores and compares the convenience of using additional signals apart from electroencephalograms. The best overall model, an ensemble of Depth-wise Separational Convolutional Neural Networks, has achieved an accuracy of 86.06%.
  arXiv  Detail & Related papers  (2021-07-23T06:37:38Z)
• Sleep Staging Based on Serialized Dual Attention Network [0.0]
  We propose a deep learning model SDAN based on raw EEG. It serially combines the channel attention and spatial attention mechanisms to filter and highlight key information. It achieves excellent results in the N1 sleep stage compared to other methods.
  arXiv  Detail & Related papers  (2021-07-18T13:18:12Z)
• Convolutional Neural Networks for Sleep Stage Scoring on a Two-Channel EEG Signal [63.18666008322476]
  Sleep problems are one of the major diseases all over the world. Basic tool used by specialists is the Polysomnogram, which is a collection of different signals recorded during sleep. Specialists have to score the different signals according to one of the standard guidelines.
  arXiv  Detail & Related papers  (2021-03-30T09:59:56Z)
• Temporal convolutional networks and transformers for classifying the sleep stage in awake or asleep using pulse oximetry signals [0.0]
  We develop a network architecture with the aim of classifying the sleep stage in awake or asleep using only HR signals from a pulse oximeter. Transformers are able to model the sequence, learning the transition rules between sleep stages. The overall accuracy, specificity, sensibility, and Cohen's Kappa coefficient were 90.0%, 94.9%, 78.1%, and 0.73.
  arXiv  Detail & Related papers  (2021-01-29T22:58:33Z)
• MSED: a multi-modal sleep event detection model for clinical sleep analysis [62.997667081978825]
  We designed a single deep neural network architecture to jointly detect sleep events in a polysomnogram. The performance of the model was quantified by F1, precision, and recall scores, and by correlating index values to clinical values.
  arXiv  Detail & Related papers  (2021-01-07T13:08:44Z)
• Brain tumor segmentation with self-ensembled, deeply-supervised 3D U-net neural networks: a BraTS 2020 challenge solution [56.17099252139182]
  We automate and standardize the task of brain tumor segmentation with U-net like neural networks. Two independent ensembles of models were trained, and each produced a brain tumor segmentation map. Our solution achieved a Dice of 0.79, 0.89 and 0.84, as well as Hausdorff 95% of 20.4, 6.7 and 19.5mm on the final test dataset.
  arXiv  Detail & Related papers  (2020-10-30T14:36:10Z)
• Automatic detection of microsleep episodes with deep learning [55.41644538483948]
  Brief fragments of sleep shorter than 15 s are defined as microsleep episodes (MSEs) maintenance of wakefulness test (MWT) is often used in a clinical setting to assess vigilance. MSEs are mostly not considered in the absence of established scoring criteria defining MSEs. We aimed for automatic detection of MSEs with machine learning based on raw EEG and EOG data as input.
  arXiv  Detail & Related papers  (2020-09-07T11:38:40Z)
• Automatic sleep stage classification with deep residual networks in a mixed-cohort setting [63.52264764099532]
  We developed a novel deep neural network model to assess the generalizability of several large-scale cohorts. Overall classification accuracy improved with increasing fractions of training data.
  arXiv  Detail & Related papers  (2020-08-21T10:48:35Z)
• Classifying sleep-wake stages through recurrent neural networks using pulse oximetry signals [0.0]
  The regulation of the autonomic nervous system changes with the sleep stages. We exploit these changes with the aim of classifying the sleep stages in awake or asleep using pulse oximeter signals. We applied a recurrent neural network to heart rate and peripheral oxygen saturation signals to classify the sleep stage every 30 seconds.
  arXiv  Detail & Related papers  (2020-08-07T21:43:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.