Monte Carlo dropout increases model repeatability

• URL: http://arxiv.org/abs/2111.06754v1
• Date: Fri, 12 Nov 2021 15:03:20 GMT
• Title: Monte Carlo dropout increases model repeatability
• Authors: Andreanne Lemay, Katharina Hoebel, Christopher P. Bridge, Didem Egemen, Ana Cecilia Rodriguez, Mark Schiffman, John Peter Campbell, Jayashree Kalpathy-Cramer
• Abstract summary: We evaluate the repeatability of four model types on images from the same patient. We study the performance of binary, multi-class, ordinal, and regression models on three medical image analysis tasks.
• Score: 2.725799462492061
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The integration of artificial intelligence into clinical workflows requires reliable and robust models. Among the main features of robustness is repeatability. Much attention is given to classification performance without assessing the model repeatability, leading to the development of models that turn out to be unusable in practice. In this work, we evaluate the repeatability of four model types on images from the same patient that were acquired during the same visit. We study the performance of binary, multi-class, ordinal, and regression models on three medical image analysis tasks: cervical cancer screening, breast density estimation, and retinopathy of prematurity classification. Moreover, we assess the impact of sampling Monte Carlo dropout predictions at test time on classification performance and repeatability. Leveraging Monte Carlo predictions significantly increased repeatability for all tasks on the binary, multi-class, and ordinal models leading to an average reduction of the 95% limits of agreement by 17% points.

Related papers

• Benchmarking foundation models as feature extractors for weakly-supervised computational pathology [0.6151041580858937]
  We benchmarked 19 histopathology foundation models on 13 patient cohorts with 6,818 patients and 9,528 slides from lung, colorectal, gastric, and breast cancers. We show that a vision-language foundation model, CONCH, yielded the highest performance when compared to vision-only foundation models, with Virchow2 as close second.
  arXiv  Detail & Related papers  (2024-08-28T14:34:45Z)
• A Comprehensive Evaluation of Histopathology Foundation Models for Ovarian Cancer Subtype Classification [1.9499122087408571]
  Histopathology foundation models show great promise across many tasks. We report the most rigorous single-task validation of histopathology foundation models to date. Histopathology foundation models offer a clear benefit to ovarian cancer subtyping.
  arXiv  Detail & Related papers  (2024-05-16T11:21:02Z)
• The effect of data augmentation and 3D-CNN depth on Alzheimer's Disease detection [51.697248252191265]
  This work summarizes and strictly observes best practices regarding data handling, experimental design, and model evaluation. We focus on Alzheimer's Disease (AD) detection, which serves as a paradigmatic example of challenging problem in healthcare. Within this framework, we train predictive 15 models, considering three different data augmentation strategies and five distinct 3D CNN architectures.
  arXiv  Detail & Related papers  (2023-09-13T10:40:41Z)
• DeepJoint: Robust Survival Modelling Under Clinical Presence Shift [2.9745607433320926]
  We propose a recurrent neural network which models three clinical presence dimensions in parallel to the survival outcome. On a prediction task, explicit modelling of these three processes showed improved performance in comparison to state-of-the-art predictive models.
  arXiv  Detail & Related papers  (2022-05-26T16:42:38Z)
• Improving the repeatability of deep learning models with Monte Carlo dropout [1.8951826092927349]
  We evaluate the repeatability of four model types (binary, multi-class, ordinal, and regression) on images that were acquired from the same patient during the same visit. We study the performance of binary, multi-class, ordinal, and regression models on four medical image classification tasks from public and private datasets.
  arXiv  Detail & Related papers  (2022-02-15T16:46:44Z)
• Multi-task fusion for improving mammography screening data classification [3.7683182861690843]
  We propose a pipeline approach, where we first train a set of individual, task-specific models. We then investigate the fusion thereof, which is in contrast to the standard model ensembling strategy. Our fusion approaches improve AUC scores significantly by up to 0.04 compared to standard model ensembling.
  arXiv  Detail & Related papers  (2021-12-01T13:56:27Z)
• A multi-stage machine learning model on diagnosis of esophageal manometry [50.591267188664666]
  The framework includes deep-learning models at the swallow-level stage and feature-based machine learning models at the study-level stage. This is the first artificial-intelligence-style model to automatically predict CC diagnosis of HRM study from raw multi-swallow data.
  arXiv  Detail & Related papers  (2021-06-25T20:09:23Z)
• On the Robustness of Pretraining and Self-Supervision for a Deep Learning-based Analysis of Diabetic Retinopathy [70.71457102672545]
  We compare the impact of different training procedures for diabetic retinopathy grading. We investigate different aspects such as quantitative performance, statistics of the learned feature representations, interpretability and robustness to image distortions. Our results indicate that models from ImageNet pretraining report a significant increase in performance, generalization and robustness to image distortions.
  arXiv  Detail & Related papers  (2021-06-25T08:32:45Z)
• Improving the Reconstruction of Disentangled Representation Learners via Multi-Stage Modeling [54.94763543386523]
  Current autoencoder-based disentangled representation learning methods achieve disentanglement by penalizing the ( aggregate) posterior to encourage statistical independence of the latent factors. We present a novel multi-stage modeling approach where the disentangled factors are first learned using a penalty-based disentangled representation learning method. Then, the low-quality reconstruction is improved with another deep generative model that is trained to model the missing correlated latent variables.
  arXiv  Detail & Related papers  (2020-10-25T18:51:15Z)
• Robust Finite Mixture Regression for Heterogeneous Targets [70.19798470463378]
  We propose an FMR model that finds sample clusters and jointly models multiple incomplete mixed-type targets simultaneously. We provide non-asymptotic oracle performance bounds for our model under a high-dimensional learning framework. The results show that our model can achieve state-of-the-art performance.
  arXiv  Detail & Related papers  (2020-10-12T03:27:07Z)
• On the model-based stochastic value gradient for continuous reinforcement learning [50.085645237597056]
  We show that simple model-based agents can outperform state-of-the-art model-free agents in terms of both sample-efficiency and final reward. Our findings suggest that model-based policy evaluation deserves closer attention.
  arXiv  Detail & Related papers  (2020-08-28T17:58:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.