The Future will be Different than Today: Model Evaluation Considerations when Developing Translational Clinical Biomarker

• URL: http://arxiv.org/abs/2107.08787v1
• Date: Tue, 13 Jul 2021 19:36:25 GMT
• Title: The Future will be Different than Today: Model Evaluation Considerations when Developing Translational Clinical Biomarker
• Authors: Yichen Lu, Jane Fridlyand, Tiffany Tang, Ting Qi, Noah Simon and Ning Leng
• Abstract summary: We present one evaluation strategy by using leave-one-study-out (LOSO) in place of conventional cross-validation (cv) methods. To demonstrate the performance of K-fold vs LOSO cv in estimating the effect size of biomarkers, we leveraged data from clinical trials and simulation studies.
• Score: 4.549866091318765
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Finding translational biomarkers stands center stage of the future of personalized medicine in healthcare. We observed notable challenges in identifying robust biomarkers as some with great performance in one scenario often fail to perform well in new trials (e.g. different population, indications). With rapid development in the clinical trial world (e.g. assay, disease definition), new trials very likely differ from legacy ones in many perspectives and in development of biomarkers this heterogeneity should be considered. In response, we recommend considering building in the heterogeneity when evaluating biomarkers. In this paper, we present one evaluation strategy by using leave-one-study-out (LOSO) in place of conventional cross-validation (cv) methods to account for the potential heterogeneity across trials used for building and testing the biomarkers. To demonstrate the performance of K-fold vs LOSO cv in estimating the effect size of biomarkers, we leveraged data from clinical trials and simulation studies. In our assessment, LOSO cv provided a more objective estimate of the future performance. This conclusion remained true across different evaluation metrics and different statistical methods.

Related papers

• FedMedICL: Towards Holistic Evaluation of Distribution Shifts in Federated Medical Imaging [68.6715007665896]
  FedMedICL is a unified framework and benchmark to holistically evaluate federated medical imaging challenges. We comprehensively evaluate several popular methods on six diverse medical imaging datasets. We find that a simple batch balancing technique surpasses advanced methods in average performance across FedMedICL experiments.
  arXiv  Detail & Related papers  (2024-07-11T19:12:23Z)
• Lifestyle-Informed Personalized Blood Biomarker Prediction via Novel Representation Learning [7.845988771273181]
  We introduce a novel framework for predicting future blood biomarker values. Our proposed method learns a similarity-based embedding space that captures the complex relationship between biomarkers and lifestyle factors.
  arXiv  Detail & Related papers  (2024-07-09T23:52:53Z)
• Enhancing predictive imaging biomarker discovery through treatment effect analysis [2.8300022117286523]
  This study focuses on the discovery of predictive imaging biomarkers, aiming to leverage pre-treatment images to unveil new causal relationships. Previous approaches relied on labor-intensive handcrafted or manually derived features, which may introduce biases. We propose an evaluation protocol for this task to assess a model's ability to identify predictive imaging biomarkers and differentiate them from prognostic ones.
  arXiv  Detail & Related papers  (2024-06-04T17:54:44Z)
• Improving Biomedical Entity Linking with Retrieval-enhanced Learning [53.24726622142558]
  $k$NN-BioEL provides a BioEL model with the ability to reference similar instances from the entire training corpus as clues for prediction. We show that $k$NN-BioEL outperforms state-of-the-art baselines on several datasets.
  arXiv  Detail & Related papers  (2023-12-15T14:04:23Z)
• Spatio-Temporal Similarity Measure based Multi-Task Learning for Predicting Alzheimer's Disease Progression using MRI Data [18.669489433316127]
  We propose a novel-temporal- similarity measure based multi-task learning approach for effectively predicting Alzheimer's disease progression. Our method also enables performing longitudinal stability selection to identify the changing relationships between biomarkers. We prove that the synergistic deteriorating biomarkers between cortical volumes or surface areas have a significant effect on the cognitive prediction.
  arXiv  Detail & Related papers  (2023-11-06T21:59:19Z)
• A marker-less human motion analysis system for motion-based biomarker discovery in knee disorders [60.99112047564336]
  The NHS has been having increased difficulty seeing all low-risk patients, this includes but not limited to suspected osteoarthritis (OA) patients. We propose a novel method of automated biomarker identification for diagnosis of knee disorders and the monitoring of treatment progression.
  arXiv  Detail & Related papers  (2023-04-26T16:47:42Z)
• Regression-based Deep-Learning predicts molecular biomarkers from pathology slides [40.24757332810004]
  We developed and evaluated a new self-supervised attention-based weakly supervised regression method that predicts continuous biomarkers directly from images. Using regression significantly enhances the accuracy of biomarker prediction, while also improving the interpretability of the results over classification. Our open-source regression approach offers a promising alternative for continuous biomarker analysis in computational pathology.
  arXiv  Detail & Related papers  (2023-04-11T11:43:51Z)
• Multi-class versus One-class classifier in spontaneous speech analysis oriented to Alzheimer Disease diagnosis [58.720142291102135]
  The aim of our project is to contribute to earlier diagnosis of AD and better estimates of its severity by using automatic analysis performed through new biomarkers extracted from speech signal. The use of information about outlier and Fractal Dimension features improves the system performance.
  arXiv  Detail & Related papers  (2022-03-21T09:57:20Z)
• Binary disease prediction using tail quantiles of the distribution of continuous biomarkers [0.0]
  In the analysis of binary disease classification, single biomarkers might not have significant discriminating power. We propose quantile based prediction (QBP), a binary classification method that is based on the selection of multiple continuous biomarkers.
  arXiv  Detail & Related papers  (2021-03-23T09:20:10Z)
• Cancer Gene Profiling through Unsupervised Discovery [49.28556294619424]
  We introduce a novel, automatic and unsupervised framework to discover low-dimensional gene biomarkers. Our method is based on the LP-Stability algorithm, a high dimensional center-based unsupervised clustering algorithm. Our signature reports promising results on distinguishing immune inflammatory and immune desert tumors.
  arXiv  Detail & Related papers  (2021-02-11T09:04:45Z)
• Personalized pathology test for Cardio-vascular disease: Approximate Bayesian computation with discriminative summary statistics learning [48.7576911714538]
  We propose a platelet deposition model and an inferential scheme to estimate the biologically meaningful parameters using approximate computation. This work opens up an unprecedented opportunity of personalized pathology test for CVD detection and medical treatment.
  arXiv  Detail & Related papers  (2020-10-13T15:20:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.