Uncertainty Quantification on Clinical Trial Outcome Prediction

• URL: http://arxiv.org/abs/2401.03482v2
• Date: Tue, 18 Jun 2024 05:09:46 GMT
• Title: Uncertainty Quantification on Clinical Trial Outcome Prediction
• Authors: Tianyi Chen, Yingzhou Lu, Nan Hao, Capucine Van Rechem, Jintai Chen, Tianfan Fu,
• Abstract summary: We propose incorporating uncertainty quantification into clinical trial outcome predictions. Our main goal is to enhance the model's ability to discern nuanced differences. We have adopted a selective classification approach to fulfill our objective.
• Score: 37.238845949535616
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The importance of uncertainty quantification is increasingly recognized in the diverse field of machine learning. Accurately assessing model prediction uncertainty can help provide deeper understanding and confidence for researchers and practitioners. This is especially critical in medical diagnosis and drug discovery areas, where reliable predictions directly impact research quality and patient health. In this paper, we proposed incorporating uncertainty quantification into clinical trial outcome predictions. Our main goal is to enhance the model's ability to discern nuanced differences, thereby significantly improving its overall performance. We have adopted a selective classification approach to fulfill our objective, integrating it seamlessly with the Hierarchical Interaction Network (HINT), which is at the forefront of clinical trial prediction modeling. Selective classification, encompassing a spectrum of methods for uncertainty quantification, empowers the model to withhold decision-making in the face of samples marked by ambiguity or low confidence, thereby amplifying the accuracy of predictions for the instances it chooses to classify. A series of comprehensive experiments demonstrate that incorporating selective classification into clinical trial predictions markedly enhances the model's performance, as evidenced by significant upticks in pivotal metrics such as PR-AUC, F1, ROC-AUC, and overall accuracy. Specifically, the proposed method achieved 32.37\%, 21.43\%, and 13.27\% relative improvement on PR-AUC over the base model (HINT) in phase I, II, and III trial outcome prediction, respectively. When predicting phase III, our method reaches 0.9022 PR-AUC scores. These findings illustrate the robustness and prospective utility of this strategy within the area of clinical trial predictions, potentially setting a new benchmark in the field.

Related papers

• Predictive uncertainty estimation in deep learning for lung carcinoma classification in digital pathology under real dataset shifts [2.309018557701645]
  This paper evaluates whether predictive uncertainty estimation adds robustness to deep learning-based diagnostic decision-making systems. We first investigate three popular methods for improving predictive uncertainty: Monte Carlo dropout, deep ensemble, and few-shot learning on lung adenocarcinoma classification as a primary disease in whole slide images.
  arXiv  Detail & Related papers  (2024-08-15T21:49:43Z)
• SepsisLab: Early Sepsis Prediction with Uncertainty Quantification and Active Sensing [67.8991481023825]
  Sepsis is the leading cause of in-hospital mortality in the USA. Existing predictive models are usually trained on high-quality data with few missing information. For the potential high-risk patients with low confidence due to limited observations, we propose a robust active sensing algorithm.
  arXiv  Detail & Related papers  (2024-07-24T04:47:36Z)
• Uncertainty estimations methods for a deep learning model to aid in clinical decision-making -- a clinician's perspective [0.0]
  There are several deep learning-inspired uncertainty estimation techniques, but few are implemented on medical datasets. We compared dropout variational inference (DO), test-time augmentation (TTA), conformal predictions, and single deterministic methods for estimating uncertainty. It may be important to evaluate multiple estimations techniques before incorporating a model into clinical practice.
  arXiv  Detail & Related papers  (2022-10-02T17:54:54Z)
• Improving Trustworthiness of AI Disease Severity Rating in Medical Imaging with Ordinal Conformal Prediction Sets [0.7734726150561088]
  A lack of statistically rigorous uncertainty quantification is a significant factor undermining trust in AI results. Recent developments in distribution-free uncertainty quantification present practical solutions for these issues. We demonstrate a technique for forming ordinal prediction sets that are guaranteed to contain the correct stenosis severity.
  arXiv  Detail & Related papers  (2022-07-05T18:01:20Z)
• Literature-Augmented Clinical Outcome Prediction [10.46990394710927]
  We introduce techniques to help bridge this gap between EBM and AI-based clinical models. We propose a novel system that automatically retrieves patient-specific literature based on intensive care (ICU) patient information. Our model is able to substantially boost predictive accuracy on three challenging tasks in comparison to strong recent baselines.
  arXiv  Detail & Related papers  (2021-11-16T11:19:02Z)
• Clinical Outcome Prediction from Admission Notes using Self-Supervised Knowledge Integration [55.88616573143478]
  Outcome prediction from clinical text can prevent doctors from overlooking possible risks. Diagnoses at discharge, procedures performed, in-hospital mortality and length-of-stay prediction are four common outcome prediction targets. We propose clinical outcome pre-training to integrate knowledge about patient outcomes from multiple public sources.
  arXiv  Detail & Related papers  (2021-02-08T10:26:44Z)
• Bayesian prognostic covariate adjustment [59.75318183140857]
  Historical data about disease outcomes can be integrated into the analysis of clinical trials in many ways. We build on existing literature that uses prognostic scores from a predictive model to increase the efficiency of treatment effect estimates.
  arXiv  Detail & Related papers  (2020-12-24T05:19:03Z)
• Increasing the efficiency of randomized trial estimates via linear adjustment for a prognostic score [59.75318183140857]
  Estimating causal effects from randomized experiments is central to clinical research. Most methods for historical borrowing achieve reductions in variance by sacrificing strict type-I error rate control.
  arXiv  Detail & Related papers  (2020-12-17T21:10:10Z)
• UNITE: Uncertainty-based Health Risk Prediction Leveraging Multi-sourced Data [81.00385374948125]
  We present UNcertaInTy-based hEalth risk prediction (UNITE) model. UNITE provides accurate disease risk prediction and uncertainty estimation leveraging multi-sourced health data. We evaluate UNITE on real-world disease risk prediction tasks: nonalcoholic fatty liver disease (NASH) and Alzheimer's disease (AD) UNITE achieves up to 0.841 in F1 score for AD detection, up to 0.609 in PR-AUC for NASH detection, and outperforms various state-of-the-art baselines by up to $19%$ over the best baseline.
  arXiv  Detail & Related papers  (2020-10-22T02:28:11Z)
• Uncertainty estimation for classification and risk prediction on medical tabular data [0.0]
  This work advances the understanding of uncertainty estimation for classification and risk prediction on medical data. In a data-scarce field such as healthcare, the ability to measure the uncertainty of a model's prediction could potentially lead to improved effectiveness of decision support tools.
  arXiv  Detail & Related papers  (2020-04-13T08:46:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.