Personalized Scorer Modeling: A Learning-Based Framework for Deriving Robust Sleep Stage Labels from Multiple Experts

• URL: http://arxiv.org/abs/2608.12446v1
• Date: Wed, 12 Aug 2026 17:28:43 GMT
• Title: Personalized Scorer Modeling: A Learning-Based Framework for Deriving Robust Sleep Stage Labels from Multiple Experts
• Abstract summary: This study investigates whether multi-scored datasets can be used to construct more reliable reference labels from the collective behavior of multiple experts.<n>We use the publicly available DOD-H and DOD-O datasets. EEG (C3-M2) and chin EMG signals were segmented into 30-s epochs, and 30 features were extracted from each modality, yielding 60 features for EEG+EMG.<n>We propose a learning-based hypnogram (LBH) that models the stage-specific behavior of each scorer using confusion matrices derived from machine-learning models.
• Score: 1.0439136407307046
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Sleep stage classification is important for the diagnosis and management of sleep disorders, yet most automatic staging studies evaluate models against a single reference hypnogram despite known inter-scorer variability. This study investigates whether multi-scored datasets can be used to construct more reliable reference labels from the collective behavior of multiple experts. We use the publicly available DOD-H and DOD-O datasets. EEG (C3-M2) and chin EMG signals were segmented into 30-s epochs, and 30 features were extracted from each modality, yielding 60 features for EEG+EMG. We propose a learning-based hypnogram (LBH) that models the stage-specific behavior of each scorer using confusion matrices derived from machine-learning models. After column normalization, these matrices estimate the probability of each true sleep stage given each scorer's label; probabilities are aggregated across scorers to assign the final label for each epoch. LBH was evaluated with random forest, support vector machine, and multilayer perceptron classifiers under EEG-only and EEG+EMG settings, and compared with the dataset hypnogram (DH) and best-scorer hypnogram (BSH). LBH consistently improved overall performance. The best results were obtained with random forest and EEG+EMG, reaching 86.07% accuracy, 85.46% precision, and 85.29% F1-score on DOD-H, and 86.04% accuracy, 85.21% precision, and 84.70% F1-score on DOD-O. These findings suggest that personalized scorer modeling can improve reference hypnogram construction without discarding information from individual experts.

Related papers

• eegFloss: A Python package for refining sleep EEG recordings using machine learning models [0.559239450391449]
  This paper introduces eegFloss, an open-source Python package to utilize eegUsability, a novel machine learning (ML) model to detect artifacts in sleep EEG recordings.<n>eegUsability has been trained and evaluated on manually artifact-labeled EEG data collected from 15 participants over 127 nights using the Zmax headband.<n>eegFloss can enhance the precision and rigor of their analysis as well as the accuracy and reliability of their outcomes.
  arXiv  Detail & Related papers  (2025-07-08T22:27:43Z)
• SLEEPYLAND: trust begins with fair evaluation of automatic sleep staging models [0.0]
  We present SLEEPYLAND, an open-source sleep staging evaluation framework.<n>It includes more than 220'000 hours in-domain (ID) sleep recordings, and more than 84'000 hours out-of-domain (OOD) sleep recordings.<n>We introduce SOMNUS, an ensemble combining models across architectures and channel setups via soft voting.
  arXiv  Detail & Related papers  (2025-06-10T08:46:19Z)
• Automatic detection of abnormal clinical EEG: comparison of a finetuned foundation model with two deep learning models [0.24848203755267906]
  We compare two deep learning models (CNN-LSTM and Transformer-based) with BioSerenity-E1, a proposed foundation model, in the task of classifying entire EEG recordings as normal or abnormal.<n>The three models were trained or finetuned on 2,500 EEG recordings.<n>Our results highlight the usefulness of leveraging pre-trained models for automatic EEG classification.
  arXiv  Detail & Related papers  (2025-05-13T07:07:24Z)
• Toward Foundational Model for Sleep Analysis Using a Multimodal Hybrid Self-Supervised Learning Framework [3.135883872525168]
  This study introduces SynthSleepNet, a multimodal hybrid self-supervised learning framework for analyzing polysomnography (PSG) data.<n> SynthSleepNet effectively integrates masked prediction and contrastive learning to leverage complementary features across multiple modalities.<n>It achieved superior performance compared to state-of-the-art methods across three downstream tasks.
  arXiv  Detail & Related papers  (2025-02-18T10:11:50Z)
• EEG-Fest: Few-shot based Attention Network for Driver's Vigilance Estimation with EEG Signals [160.57870373052577]
  A lack of driver's vigilance is the main cause of most vehicle crashes. EEG has been reliable and efficient tool for drivers' drowsiness estimation.
  arXiv  Detail & Related papers  (2022-11-07T21:35:08Z)
• Examining Uniqueness and Permanence of the WAY EEG GAL dataset toward User Authentication [0.0]
  This study evaluates the discriminating capacity (uniqueness) of the EEG data from the WAY EEG GAL public dataset to authenticate individuals against one another. A feature importance algorithm is utilized to select the best features for each user to authenticate them against all others. High performing individuals with 95.3% accuracy (95.3% F1 score) using linear SVM and 97.4% accuracy (97.3% F1 score) using non-linear SVM are also observed.
  arXiv  Detail & Related papers  (2022-09-11T06:53:51Z)
• Generalizing electrocardiogram delineation: training convolutional neural networks with synthetic data augmentation [63.51064808536065]
  Existing databases for ECG delineation are small, being insufficient in size and in the array of pathological conditions they represent. This article delves has two main contributions. First, a pseudo-synthetic data generation algorithm was developed, based in probabilistically composing ECG traces given "pools" of fundamental segments, as cropped from the original databases, and a set of rules for their arrangement into coherent synthetic traces. Second, two novel segmentation-based loss functions have been developed, which attempt at enforcing the prediction of an exact number of independent structures and at producing closer segmentation boundaries by focusing on a reduced number of samples.
  arXiv  Detail & Related papers  (2021-11-25T10:11:41Z)
• Ensemble of Convolution Neural Networks on Heterogeneous Signals for Sleep Stage Scoring [63.30661835412352]
  This paper explores and compares the convenience of using additional signals apart from electroencephalograms. The best overall model, an ensemble of Depth-wise Separational Convolutional Neural Networks, has achieved an accuracy of 86.06%.
  arXiv  Detail & Related papers  (2021-07-23T06:37:38Z)
• Convolutional Neural Networks for Sleep Stage Scoring on a Two-Channel EEG Signal [63.18666008322476]
  Sleep problems are one of the major diseases all over the world. Basic tool used by specialists is the Polysomnogram, which is a collection of different signals recorded during sleep. Specialists have to score the different signals according to one of the standard guidelines.
  arXiv  Detail & Related papers  (2021-03-30T09:59:56Z)
• MSED: a multi-modal sleep event detection model for clinical sleep analysis [62.997667081978825]
  We designed a single deep neural network architecture to jointly detect sleep events in a polysomnogram. The performance of the model was quantified by F1, precision, and recall scores, and by correlating index values to clinical values.
  arXiv  Detail & Related papers  (2021-01-07T13:08:44Z)
• Multilabel 12-Lead Electrocardiogram Classification Using Gradient Boosting Tree Ensemble [64.29529357862955]
  We build an algorithm using gradient boosted tree ensembles fitted on morphology and signal processing features to classify ECG diagnosis. For each lead, we derive features from heart rate variability, PQRST template shape, and the full signal waveform. We join the features of all 12 leads to fit an ensemble of gradient boosting decision trees to predict probabilities of ECG instances belonging to each class.
  arXiv  Detail & Related papers  (2020-10-21T18:11:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.