Performance of weakly-supervised electronic health record-based phenotyping methods in rare-outcome settings

• URL: http://arxiv.org/abs/2604.09913v1
• Date: Fri, 10 Apr 2026 21:16:44 GMT
• Title: Performance of weakly-supervised electronic health record-based phenotyping methods in rare-outcome settings
• Abstract summary: weakly-supervised prediction methods, which use silver-standard labels, perform well in rare-outcome settings like vaccine safety studies.<n>Care should be taken when using weakly-supervised prediction methods in rare-outcome settings.
• Score: 1.0832844764942349
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Accurately identifying patients with specific medical conditions is a key challenge when using clinical data from electronic health records. Our objective was to comprehensively assess when weakly-supervised prediction methods, which use silver-standard labels (proxy measures of the true outcome) rather than gold-standard true labels, perform well in rare-outcome settings like vaccine safety studies. We compared three methods (PheNorm, MAP, and sureLDA) that combine structured features and features derived from clinical text using natural language processing, through an extensive simulation study with data-generating mechanisms ranging from simple to complex, varying outcome rates, and varying degrees of informative silver labels. We also considered using predicted probabilities to design a chart review validation study. No single method dominated the other across all prediction performance metrics. Probability-guided sampling selected a cohort enriched for patients with more mentions of important concepts in chart notes. SureLDA, the most complex of the three algorithms we considered, often performed well in simulations. Performance depended greatly on selected tuning parameters. Care should be taken when using weakly-supervised prediction methods in rare-outcome settings, particularly if the probabilities will be used in downstream analysis, but these methods can work well when silver labels are strong predictors of true outcomes.

Related papers

• Beyond Perfect Scores: Proof-by-Contradiction for Trustworthy Machine Learning [0.0]
  It is often unclear whether a model relies on true clinical cues or on spurious correlations in the data.<n>This paper introduces a simple yet broadly applicable trustworthiness test grounded in proof-by-contradiction.<n>Our approach trains and tests on spurious labels carefully permuted based on a potential outcomes framework.
  arXiv  Detail & Related papers  (2026-01-10T22:08:14Z)
• Uncertainty-Guided Model Selection for Tabular Foundation Models in Biomolecule Efficacy Prediction [3.108481950101193]
  In this study, we investigate an uncertainty-guided strategy for model selection.<n>We show that a TabPFN model using straightforward sequence-based features can surpass specialized state-of-the-art predictors.
  arXiv  Detail & Related papers  (2025-10-02T18:33:19Z)
• A chart review process aided by natural language processing and multi-wave adaptive sampling to expedite validation of code-based algorithms for large database studies [3.452275310496547]
  We describe an expedited process that reduces time spent by human reviewers to review each chart.<n>We illustrate this process in a case study that validates the performance of a claims-based outcome algorithm for intentional self-harm.
  arXiv  Detail & Related papers  (2025-07-25T17:11:22Z)
• AUTOCT: Automating Interpretable Clinical Trial Prediction with LLM Agents [47.640779069547534]
  AutoCT is a novel framework that combines the reasoning capabilities of large language models with the explainability of classical machine learning.<n>We show that AutoCT performs on par with or better than SOTA methods on clinical trial prediction tasks within only a limited number of self-refinement iterations.
  arXiv  Detail & Related papers  (2025-06-04T11:50:55Z)
• A Copula Based Supervised Filter for Feature Selection in Diabetes Risk Prediction Using Machine Learning [0.0]
  We propose a computationally efficient supervised filter that ranks features using the Gumbel copula upper tail dependence coefficient ($lambda_U$)<n>We benchmarked against Mutual Information, mRMR, ReliefF, and $L_1$ Elastic Net across four classifiers on two diabetes datasets.<n>We conclude that copula based feature selection via upper tail dependence is a powerful, efficient, and interpretable approach for building risk models in public health and clinical medicine.
  arXiv  Detail & Related papers  (2025-05-28T16:34:58Z)
• Curating Naturally Adversarial Datasets for Learning-Enabled Medical Cyber-Physical Systems [5.349773727704873]
  Existing research focuses on robustness to synthetic adversarial examples, crafted by adding imperceptible perturbations to clean input data. We propose a method to curate datasets comprised of natural adversarial examples to evaluate model robustness.
  arXiv  Detail & Related papers  (2023-09-01T15:52:32Z)
• TREEMENT: Interpretable Patient-Trial Matching via Personalized Dynamic Tree-Based Memory Network [54.332862955411656]
  Clinical trials are critical for drug development but often suffer from expensive and inefficient patient recruitment. In recent years, machine learning models have been proposed for speeding up patient recruitment via automatically matching patients with clinical trials. We introduce a dynamic tree-based memory network model named TREEMENT to provide accurate and interpretable patient trial matching.
  arXiv  Detail & Related papers  (2023-07-19T12:35:09Z)
• ASPEST: Bridging the Gap Between Active Learning and Selective Prediction [56.001808843574395]
  Selective prediction aims to learn a reliable model that abstains from making predictions when uncertain. Active learning aims to lower the overall labeling effort, and hence human dependence, by querying the most informative examples. In this work, we introduce a new learning paradigm, active selective prediction, which aims to query more informative samples from the shifted target domain.
  arXiv  Detail & Related papers  (2023-04-07T23:51:07Z)
• Modeling Disagreement in Automatic Data Labelling for Semi-Supervised Learning in Clinical Natural Language Processing [2.016042047576802]
  We investigate the quality of uncertainty estimates from a range of current state-of-the-art predictive models applied to the problem of observation detection in radiology reports.
  arXiv  Detail & Related papers  (2022-05-29T20:20:49Z)
• Robust and Agnostic Learning of Conditional Distributional Treatment Effects [44.31792000298105]
  We provide a new robust and model-agnostic methodology for learning the conditional DTE (CDTE) for a class of problems.<n>Our method is model-agnostic in that it can provide the best projection of CDTE onto the regression model class.<n>We investigate the behavior of our proposal in simulations, as well as in a case study of 401(k) eligibility effects on wealth.
  arXiv  Detail & Related papers  (2022-05-23T17:40:31Z)
• Active learning for medical code assignment [55.99831806138029]
  We demonstrate the effectiveness of Active Learning (AL) in multi-label text classification in the clinical domain. We apply a set of well-known AL methods to help automatically assign ICD-9 codes on the MIMIC-III dataset. Our results show that the selection of informative instances provides satisfactory classification with a significantly reduced training set.
  arXiv  Detail & Related papers  (2021-04-12T18:11:17Z)
• Bootstrapping Your Own Positive Sample: Contrastive Learning With Electronic Health Record Data [62.29031007761901]
  This paper proposes a novel contrastive regularized clinical classification model. We introduce two unique positive sampling strategies specifically tailored for EHR data. Our framework yields highly competitive experimental results in predicting the mortality risk on real-world COVID-19 EHR data.
  arXiv  Detail & Related papers  (2021-04-07T06:02:04Z)
• Learn what you can't learn: Regularized Ensembles for Transductive Out-of-distribution Detection [76.39067237772286]
  We show that current out-of-distribution (OOD) detection algorithms for neural networks produce unsatisfactory results in a variety of OOD detection scenarios. This paper studies how such "hard" OOD scenarios can benefit from adjusting the detection method after observing a batch of the test data. We propose a novel method that uses an artificial labeling scheme for the test data and regularization to obtain ensembles of models that produce contradictory predictions only on the OOD samples in a test batch.
  arXiv  Detail & Related papers  (2020-12-10T16:55:13Z)
• Hemogram Data as a Tool for Decision-making in COVID-19 Management: Applications to Resource Scarcity Scenarios [62.997667081978825]
  COVID-19 pandemics has challenged emergency response systems worldwide, with widespread reports of essential services breakdown and collapse of health care structure. This work describes a machine learning model derived from hemogram exam data performed in symptomatic patients. Proposed models can predict COVID-19 qRT-PCR results in symptomatic individuals with high accuracy, sensitivity and specificity.
  arXiv  Detail & Related papers  (2020-05-10T01:45:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.