Towards trustworthy seizure onset detection using workflow notes

• URL: http://arxiv.org/abs/2306.08728v1
• Date: Wed, 14 Jun 2023 20:13:24 GMT
• Title: Towards trustworthy seizure onset detection using workflow notes
• Abstract summary: We propose to leverage annotations that are produced by healthcare personnel in routine clinical. We show that by scaling training data to an unprecedented level of 68,920 EEG hours, seizure onset detection performance significantly improves. We also train a multilabel model that classifies 26 attributes other than seizures, such as spikes, slowing, and movement artifacts.
• Score: 5.536372101225628
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A major barrier to deploying healthcare AI models is their trustworthiness. One form of trustworthiness is a model's robustness across different subgroups: while existing models may exhibit expert-level performance on aggregate metrics, they often rely on non-causal features, leading to errors in hidden subgroups. To take a step closer towards trustworthy seizure onset detection from EEG, we propose to leverage annotations that are produced by healthcare personnel in routine clinical workflows -- which we refer to as workflow notes -- that include multiple event descriptions beyond seizures. Using workflow notes, we first show that by scaling training data to an unprecedented level of 68,920 EEG hours, seizure onset detection performance significantly improves (+12.3 AUROC points) compared to relying on smaller training sets with expensive manual gold-standard labels. Second, we reveal that our binary seizure onset detection model underperforms on clinically relevant subgroups (e.g., up to a margin of 6.5 AUROC points between pediatrics and adults), while having significantly higher false positives on EEG clips showing non-epileptiform abnormalities compared to any EEG clip (+19 FPR points). To improve model robustness to hidden subgroups, we train a multilabel model that classifies 26 attributes other than seizures, such as spikes, slowing, and movement artifacts. We find that our multilabel model significantly improves overall seizure onset detection performance (+5.9 AUROC points) while greatly improving performance among subgroups (up to +8.3 AUROC points), and decreases false positives on non-epileptiform abnormalities by 8 FPR points. Finally, we propose a clinical utility metric based on false positives per 24 EEG hours and find that our multilabel model improves this clinical utility metric by a factor of 2x across different clinical settings.

Related papers

• Beyond Scalar Scores: Exploring LLM-based Metrics for Clinical Significance Evaluation in Radiology Reports [49.5225801722164]
  Existing metrics obscure this requirement by reducing report quality to a medically ungrounded scalar.<n>We study this boundary using ReEvalMed benchmark as testbed and evaluate metric-level clinical significance.
  arXiv  Detail & Related papers  (2026-06-17T08:10:30Z)
• Improving Automated Wound Assessment Using Joint Boundary Segmentation and Multi-Class Classification Models [1.1432909951914676]
  This study presents a deep learning model based on YOLOv11 that simultaneously performs wound boundary segmentation (WBS) and wound classification (WC) across five clinically relevant wound types.<n>The results confirm the model's robustness against complex backgrounds and high intra-class variability.
  arXiv  Detail & Related papers  (2026-03-28T16:11:31Z)
• Med-SegLens: Latent-Level Model Diffing for Interpretable Medical Image Segmentation [0.0]
  We introduce Med-SegLens, a model-diffing framework that decomposes segmentation model activations into interpretable latent features.<n>We show that latent-level interventions can correct errors and improve cross-dataset adaption without retraining.
  arXiv  Detail & Related papers  (2026-02-11T04:15:36Z)
• AgentScore: Autoformulation of Deployable Clinical Scoring Systems [45.88028371034407]
  We introduce AgentScore, which performs semantically guided optimization in unit-weighted clinical checklists.<n>AgentScore outperforms existing score-generation methods and achieves AUC comparable to more flexible interpretable models.<n>On two additional externally validated tasks, AgentScore achieves higher discrimination than established guideline-based scores.
  arXiv  Detail & Related papers  (2026-01-29T21:11:06Z)
• Scalably Enhancing the Clinical Validity of a Task Benchmark with Physician Oversight [5.202988483354374]
  In this work, we propose viewing benchmarks for complex tasks as ''in-progress living documents'' that should be periodically re-evaluated.<n>We introduce a systematic, physician-in-the-loop pipeline that leverages advanced agentic verifiers to audit and relabel MedCalc-Bench.<n>Our audit reveals that a notable fraction of original labels diverge from medical ground truth due to extraction errors, calculator logic mismatches, and clinical ambiguity.
  arXiv  Detail & Related papers  (2025-12-22T18:59:34Z)
• EchoBench: Benchmarking Sycophancy in Medical Large Vision-Language Models [82.43729208063468]
  Recent benchmarks for medical Large Vision-Language Models (LVLMs) emphasize leaderboard accuracy, overlooking reliability and safety.<n>We study sycophancy -- models' tendency to uncritically echo user-provided information.<n>We introduce EchoBench, a benchmark to systematically evaluate sycophancy in medical LVLMs.
  arXiv  Detail & Related papers  (2025-09-24T14:09:55Z)
• Are Anomaly Scores Telling the Whole Story? A Benchmark for Multilevel Anomaly Detection [46.244213695024]
  Anomaly detection (AD) is a machine learning task that identifies anomalies by learning patterns from normal training data. Existing models primarily operate in a binary setting, and the anomaly scores they produce are usually based on the deviation of data points from normal data. We propose a novel setting, Multilevel AD (MAD), in which the anomaly score represents the severity of anomalies in real-world applications. Second, we introduce a novel benchmark, MAD-Bench, that evaluates models not only on their ability to detect anomalies, but also on how effectively their anomaly scores reflect severity.
  arXiv  Detail & Related papers  (2024-11-21T14:18:37Z)
• Weakly supervised deep learning model with size constraint for prostate cancer detection in multiparametric MRI and generalization to unseen domains [0.90668179713299]
  We show that the model achieves on-par performance with strong fully supervised baseline models. We also observe a performance decrease for both fully supervised and weakly supervised models when tested on unseen data domains.
  arXiv  Detail & Related papers  (2024-11-04T12:24:33Z)
• Multi-stream deep learning framework to predict mild cognitive impairment with Rey Complex Figure Test [10.324611550865926]
  We developed a multi-stream deep learning framework that integrates two distinct processing streams. The proposed multi-stream model demonstrated superior performance over baseline models in external validation. Our model has practical implications for clinical settings, where it could serve as a cost-effective tool for early screening.
  arXiv  Detail & Related papers  (2024-09-04T17:08:04Z)
• Improving Diffusion Models for ECG Imputation with an Augmented Template Prior [43.6099225257178]
  noisy and poor-quality recordings are a major issue for signals collected using mobile health systems. Recent studies have explored the imputation of missing values in ECG with probabilistic time-series models. We present a template-guided denoising diffusion probabilistic model (DDPM), PulseDiff, which is conditioned on an informative prior for a range of health conditions.
  arXiv  Detail & Related papers  (2023-10-24T11:34:15Z)
• How Does Pruning Impact Long-Tailed Multi-Label Medical Image Classifiers? [49.35105290167996]
  Pruning has emerged as a powerful technique for compressing deep neural networks, reducing memory usage and inference time without significantly affecting overall performance. This work represents a first step toward understanding the impact of pruning on model behavior in deep long-tailed, multi-label medical image classification.
  arXiv  Detail & Related papers  (2023-08-17T20:40:30Z)
• Learning Clinical Concepts for Predicting Risk of Progression to Severe COVID-19 [17.781861866125023]
  Using data from a major healthcare provider, we develop survival models predicting severe COVID-19 progression. We develop two sets of high-performance risk scores: (i) an unconstrained model built from all available features; and (ii) a pipeline that learns a small set of clinical concepts before training a risk predictor.
  arXiv  Detail & Related papers  (2022-08-28T02:59:35Z)
• SOUL: An Energy-Efficient Unsupervised Online Learning Seizure Detection Classifier [68.8204255655161]
  Implantable devices that record neural activity and detect seizures have been adopted to issue warnings or trigger neurostimulation to suppress seizures. For an implantable seizure detection system, a low power, at-the-edge, online learning algorithm can be employed to dynamically adapt to neural signal drifts. SOUL was fabricated in TSMC's 28 nm process occupying 0.1 mm2 and achieves 1.5 nJ/classification energy efficiency, which is at least 24x more efficient than state-of-the-art.
  arXiv  Detail & Related papers  (2021-10-01T23:01:20Z)
• Dual-Consistency Semi-Supervised Learning with Uncertainty Quantification for COVID-19 Lesion Segmentation from CT Images [49.1861463923357]
  We propose an uncertainty-guided dual-consistency learning network (UDC-Net) for semi-supervised COVID-19 lesion segmentation from CT images. Our proposed UDC-Net improves the fully supervised method by 6.3% in Dice and outperforms other competitive semi-supervised approaches by significant margins.
  arXiv  Detail & Related papers  (2021-04-07T16:23:35Z)
• Bootstrapping Your Own Positive Sample: Contrastive Learning With Electronic Health Record Data [62.29031007761901]
  This paper proposes a novel contrastive regularized clinical classification model. We introduce two unique positive sampling strategies specifically tailored for EHR data. Our framework yields highly competitive experimental results in predicting the mortality risk on real-world COVID-19 EHR data.
  arXiv  Detail & Related papers  (2021-04-07T06:02:04Z)
• Convolutional Neural Networks for Sleep Stage Scoring on a Two-Channel EEG Signal [63.18666008322476]
  Sleep problems are one of the major diseases all over the world. Basic tool used by specialists is the Polysomnogram, which is a collection of different signals recorded during sleep. Specialists have to score the different signals according to one of the standard guidelines.
  arXiv  Detail & Related papers  (2021-03-30T09:59:56Z)
• Severity Quantification and Lesion Localization of COVID-19 on CXR using Vision Transformer [25.144248675578286]
  Under the global pandemic of COVID-19, building an automated framework that quantifies the severity of COVID-19 has become increasingly important. We propose a novel Vision Transformer tailored for both quantification of the severity and clinically applicable localization of the COVID-19 related lesions. Our model is trained in a weakly-supervised manner to generate the full probability maps from weak array-based labels.
  arXiv  Detail & Related papers  (2021-03-12T03:17:19Z)
• Many-to-One Distribution Learning and K-Nearest Neighbor Smoothing for Thoracic Disease Identification [83.6017225363714]
  deep learning has become the most powerful computer-aided diagnosis technology for improving disease identification performance. For chest X-ray imaging, annotating large-scale data requires professional domain knowledge and is time-consuming. In this paper, we propose many-to-one distribution learning (MODL) and K-nearest neighbor smoothing (KNNS) methods to improve a single model's disease identification performance.
  arXiv  Detail & Related papers  (2021-02-26T02:29:30Z)
• EventScore: An Automated Real-time Early Warning Score for Clinical Events [3.3039612529376625]
  We build an interpretable model for the early prediction of various adverse clinical events indicative of clinical deterioration. The model is evaluated on two datasets and four clinical events. Our model can be entirely automated without requiring any manually recorded features.
  arXiv  Detail & Related papers  (2021-02-11T11:55:08Z)
• Self-Training with Improved Regularization for Sample-Efficient Chest X-Ray Classification [80.00316465793702]
  We present a deep learning framework that enables robust modeling in challenging scenarios. Our results show that using 85% lesser labeled data, we can build predictive models that match the performance of classifiers trained in a large-scale data setting.
  arXiv  Detail & Related papers  (2020-05-03T02:36:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.