Bridging AI and Clinical Practice: Integrating Automated Sleep Scoring Algorithm with Uncertainty-Guided Physician Review

• URL: http://arxiv.org/abs/2312.14996v1
• Date: Fri, 22 Dec 2023 15:58:09 GMT
• Title: Bridging AI and Clinical Practice: Integrating Automated Sleep Scoring Algorithm with Uncertainty-Guided Physician Review
• Authors: Michal Bechny (1 and 2), Giuliana Monachino (1 and 2), Luigi Fiorillo (2), Julia van der Meer (3), Markus H. Schmidt (3 and 4), Claudio L. A. Bassetti (3), Athina Tzovara (1 and 5), Francesca D. Faraci (2) ((1) Institute of Computer Science, University of Bern, Bern, Switzerland (2) Institute of Digital Technologies for Personalized Healthcare (MeDiTech), University of Applied Sciences and Arts of Southern Switzerland, Lugano, Switzerland (3) Department of Neurology, Inselspital, Bern University Hospital, University of Bern, Bern, Switzerland (4) Ohio Sleep Medicine Institute, Dublin, United States (5) Center for Experimental Neurology, Department of Neurology, Inselspital, Bern University Hospital, University of Bern, Bern, Switzerland)
• Abstract summary: This study aims to enhance the clinical use of automated sleep-scoring algorithms by incorporating an uncertainty estimation approach. Total of 19578 PSGs from 13 open-access databases were used to train U-Sleep, a state-of-the-art sleep-scoring algorithm.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Purpose: This study aims to enhance the clinical use of automated sleep-scoring algorithms by incorporating an uncertainty estimation approach to efficiently assist clinicians in the manual review of predicted hypnograms, a necessity due to the notable inter-scorer variability inherent in polysomnography (PSG) databases. Our efforts target the extent of review required to achieve predefined agreement levels, examining both in-domain and out-of-domain data, and considering subjects diagnoses. Patients and methods: Total of 19578 PSGs from 13 open-access databases were used to train U-Sleep, a state-of-the-art sleep-scoring algorithm. We leveraged a comprehensive clinical database of additional 8832 PSGs, covering a full spectrum of ages and sleep-disorders, to refine the U-Sleep, and to evaluate different uncertainty-quantification approaches, including our novel confidence network. The ID data consisted of PSGs scored by over 50 physicians, and the two OOD sets comprised recordings each scored by a unique senior physician. Results: U-Sleep demonstrated robust performance, with Cohen's kappa (K) at 76.2% on ID and 73.8-78.8% on OOD data. The confidence network excelled at identifying uncertain predictions, achieving AUROC scores of 85.7% on ID and 82.5-85.6% on OOD data. Independently of sleep-disorder status, statistical evaluations revealed significant differences in confidence scores between aligning vs discording predictions, and significant correlations of confidence scores with classification performance metrics. To achieve K of at least 90% with physician intervention, examining less than 29.0% of uncertain epochs was required, substantially reducing physicians workload, and facilitating near-perfect agreement.

Related papers

• Using Pre-training and Interaction Modeling for ancestry-specific disease prediction in UK Biobank [69.90493129893112]
  Recent genome-wide association studies (GWAS) have uncovered the genetic basis of complex traits, but show an under-representation of non-European descent individuals. Here, we assess whether we can improve disease prediction across diverse ancestries using multiomic data.
  arXiv  Detail & Related papers  (2024-04-26T16:39:50Z)
• Quantifying Impairment and Disease Severity Using AI Models Trained on Healthy Subjects [27.786240241494436]
  COnfidence-Based chaRacterization of Anomalies (COBRA) score exploits the decrease in confidence of these models when presented with impaired or diseased patients. We applied the COBRA score to address a key limitation of current clinical evaluation of upper-body impairment in stroke patients.
  arXiv  Detail & Related papers  (2023-11-21T18:45:52Z)
• A Federated Learning Framework for Stenosis Detection [70.27581181445329]
  This study explores the use of Federated Learning (FL) for stenosis detection in coronary angiography images (CA) Two heterogeneous datasets from two institutions were considered: dataset 1 includes 1219 images from 200 patients, which we acquired at the Ospedale Riuniti of Ancona (Italy) dataset 2 includes 7492 sequential images from 90 patients from a previous study available in the literature.
  arXiv  Detail & Related papers  (2023-10-30T11:13:40Z)
• Real-World Performance of Autonomously Reporting Normal Chest Radiographs in NHS Trusts Using a Deep-Learning Algorithm on the GP Pathway [0.0]
  A DL algorithm has been deployed in Somerset NHS Foundation Trust (SFT) and Calderdale & Huddersfield NHS Trust (CHFT) since December 2022. The algorithm was developed and trained prior to deployment, and is used to assign abnormality scores to each GP-requested chest x-ray (CXR) The algorithm classifies a subset of examinations with the lowest abnormality scores as High Confidence Normal (HCN)
  arXiv  Detail & Related papers  (2023-06-28T11:34:42Z)
• MedLens: Improve Mortality Prediction Via Medical Signs Selecting and Regression [4.43322868663347]
  Data-quality problem of original clinical signs is less discussed in the literature. We designed MEDLENS, with an automatic vital medical signs selection approach via statistics and a flexible approach for high missing rate time series. It achieves a very high accuracy performance of 0.96 AUC-ROC and 0.81 AUC-PR, which exceeds the previous benchmark.
  arXiv  Detail & Related papers  (2023-05-19T15:28:02Z)
• Towards Reliable Medical Image Segmentation by utilizing Evidential Calibrated Uncertainty [52.03490691733464]
  We introduce DEviS, an easily implementable foundational model that seamlessly integrates into various medical image segmentation networks. By leveraging subjective logic theory, we explicitly model probability and uncertainty for the problem of medical image segmentation. DeviS incorporates an uncertainty-aware filtering module, which utilizes the metric of uncertainty-calibrated error to filter reliable data.
  arXiv  Detail & Related papers  (2023-01-01T05:02:46Z)
• MOMO -- Deep Learning-driven classification of external DICOM studies for PACS archivation [0.9498643829295902]
  MOMO (MOdality Mapping and Orchestration) is a deep learning-based approach to automate this mapping process. A set of 11,934 imaging series with existing labels was retrieved from the local hospital's PACS database to train the neural networks. MOMO outperforms either by a large margin in accuracy and with increased predictive power (99.29% predictive power, 92.71% accuracy, 2.63% minor errors)
  arXiv  Detail & Related papers  (2021-12-01T17:16:41Z)
• Advancing COVID-19 Diagnosis with Privacy-Preserving Collaboration in Artificial Intelligence [79.038671794961]
  We launch the Unified CT-COVID AI Diagnostic Initiative (UCADI), where the AI model can be distributedly trained and independently executed at each host institution. Our study is based on 9,573 chest computed tomography scans (CTs) from 3,336 patients collected from 23 hospitals located in China and the UK.
  arXiv  Detail & Related papers  (2021-11-18T00:43:41Z)
• Pediatric Automatic Sleep Staging: A comparative study of state-of-the-art deep learning methods [16.651453507701966]
  We conduct a large-scale comparative study on the state-of-the-art deep learning methods for pediatric automatic sleep staging. A selection of six different deep neural networks with diverging features are adopted to evaluate a sample of more than 1,200 children. Experiments show that the performance of automated pediatric sleep staging when evaluated on new subjects is equivalent to the expert-level one reported on adults.
  arXiv  Detail & Related papers  (2021-08-23T15:39:48Z)
• UNITE: Uncertainty-based Health Risk Prediction Leveraging Multi-sourced Data [81.00385374948125]
  We present UNcertaInTy-based hEalth risk prediction (UNITE) model. UNITE provides accurate disease risk prediction and uncertainty estimation leveraging multi-sourced health data. We evaluate UNITE on real-world disease risk prediction tasks: nonalcoholic fatty liver disease (NASH) and Alzheimer's disease (AD) UNITE achieves up to 0.841 in F1 score for AD detection, up to 0.609 in PR-AUC for NASH detection, and outperforms various state-of-the-art baselines by up to $19%$ over the best baseline.
  arXiv  Detail & Related papers  (2020-10-22T02:28:11Z)
• COMPOSE: Cross-Modal Pseudo-Siamese Network for Patient Trial Matching [70.08786840301435]
  We propose CrOss-Modal PseudO-SiamEse network (COMPOSE) to address these challenges for patient-trial matching. Experiment results show COMPOSE can reach 98.0% AUC on patient-criteria matching and 83.7% accuracy on patient-trial matching.
  arXiv  Detail & Related papers  (2020-06-15T21:01:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.