A Quantitative Comparison of Epistemic Uncertainty Maps Applied to Multi-Class Segmentation

• URL: http://arxiv.org/abs/2109.10702v1
• Date: Wed, 22 Sep 2021 12:48:19 GMT
• Title: A Quantitative Comparison of Epistemic Uncertainty Maps Applied to Multi-Class Segmentation
• Authors: Robin Camarasa (1 and 2), Daniel Bos (2 and 3), Jeroen Hendrikse (4), Paul Nederkoorn (5), M. Eline Kooi (6), Aad van der Lugt (2), Marleen de Bruijne (1, 2 and 7), ((1) Biomedical Imaging Group Rotterdam, Department of Radiology and Nuclear Medicine, Erasmus MC, Rotterdam, The Netherlands, (2) Department of Radiology and Nuclear Medicine, Erasmus MC, Rotterdam, The Netherlands, (3) Department of Epidemiology, Erasmus MC, Rotterdam, The Netherlands, (4) Department of Radiology, University Medical Center Utrecht, Utrecht, The Netherlands, (5) Department of Neurology, Academic Medical Center University of Amsterdam, Amsterdam, The Netherlands, (6) Department of Radiology and Nuclear Medicine, CARIM School for Cardiovascular Diseases, Maastricht University Medical Center, Maastricht, The Netherlands, (7) Department of Computer Science, University of Copenhagen, Denmark)
• Abstract summary: This paper highlights a systematic approach to define and quantitatively compare those methods in two different contexts. We applied this analysis to a multi-class segmentation of the carotid artery lumens and vessel wall, on a multi-center, multi-scanner, multi-sequence dataset. We made a python package available to reproduce our analysis on different data and tasks.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Uncertainty assessment has gained rapid interest in medical image analysis. A popular technique to compute epistemic uncertainty is the Monte-Carlo (MC) dropout technique. From a network with MC dropout and a single input, multiple outputs can be sampled. Various methods can be used to obtain epistemic uncertainty maps from those multiple outputs. In the case of multi-class segmentation, the number of methods is even larger as epistemic uncertainty can be computed voxelwise per class or voxelwise per image. This paper highlights a systematic approach to define and quantitatively compare those methods in two different contexts: class-specific epistemic uncertainty maps (one value per image, voxel and class) and combined epistemic uncertainty maps (one value per image and voxel). We applied this quantitative analysis to a multi-class segmentation of the carotid artery lumen and vessel wall, on a multi-center, multi-scanner, multi-sequence dataset of (MR) images. We validated our analysis over 144 sets of hyperparameters of a model. Our main analysis considers the relationship between the order of the voxels sorted according to their epistemic uncertainty values and the misclassification of the prediction. Under this consideration, the comparison of combined uncertainty maps reveals that the multi-class entropy and the multi-class mutual information statistically out-perform the other combined uncertainty maps under study. In a class-specific scenario, the one-versus-all entropy statistically out-performs the class-wise entropy, the class-wise variance and the one versus all mutual information. The class-wise entropy statistically out-performs the other class-specific uncertainty maps in terms of calibration. We made a python package available to reproduce our analysis on different data and tasks.

Related papers

• Predictive uncertainty estimation in deep learning for lung carcinoma classification in digital pathology under real dataset shifts [2.309018557701645]
  This paper evaluates whether predictive uncertainty estimation adds robustness to deep learning-based diagnostic decision-making systems. We first investigate three popular methods for improving predictive uncertainty: Monte Carlo dropout, deep ensemble, and few-shot learning on lung adenocarcinoma classification as a primary disease in whole slide images.
  arXiv  Detail & Related papers  (2024-08-15T21:49:43Z)
• Seeing Unseen: Discover Novel Biomedical Concepts via Geometry-Constrained Probabilistic Modeling [53.7117640028211]
  We present a geometry-constrained probabilistic modeling treatment to resolve the identified issues. We incorporate a suite of critical geometric properties to impose proper constraints on the layout of constructed embedding space. A spectral graph-theoretic method is devised to estimate the number of potential novel classes.
  arXiv  Detail & Related papers  (2024-03-02T00:56:05Z)
• Probabilistic Modeling of Inter- and Intra-observer Variability in Medical Image Segmentation [12.594098548008832]
  We propose a novel model, called Probabilistic Inter-Observer and iNtra-Observer variation NetwOrk (Pionono) It captures the labeling behavior of each rater with a multidimensional probability distribution to produce probabilistic segmentation predictions. Experiments on real-world cancer segmentation datasets demonstrate the high accuracy and efficiency of Pionono.
  arXiv  Detail & Related papers  (2023-07-21T07:29:38Z)
• Benchmarking common uncertainty estimation methods with histopathological images under domain shift and label noise [62.997667081978825]
  In high-risk environments, deep learning models need to be able to judge their uncertainty and reject inputs when there is a significant chance of misclassification. We conduct a rigorous evaluation of the most commonly used uncertainty and robustness methods for the classification of Whole Slide Images. We observe that ensembles of methods generally lead to better uncertainty estimates as well as an increased robustness towards domain shifts and label noise.
  arXiv  Detail & Related papers  (2023-01-03T11:34:36Z)
• Calibrating Ensembles for Scalable Uncertainty Quantification in Deep Learning-based Medical Segmentation [0.42008820076301906]
  Uncertainty quantification in automated image analysis is highly desired in many applications. Current uncertainty quantification approaches do not scale well in high-dimensional real-world problems. We propose a scalable and intuitive framework to calibrate ensembles of deep learning models to produce uncertainty quantification measurements.
  arXiv  Detail & Related papers  (2022-09-20T09:09:48Z)
• A Multiple kernel testing procedure for non-proportional hazards in factorial designs [4.358626952482687]
  We propose a Multiple kernel testing procedure to infer survival data when several factors are of interest simultaneously. Our method is able to deal with complex data and can be seen as an alternative to the omnipresent Cox model when assumptions such as proportionality cannot be justified.
  arXiv  Detail & Related papers  (2022-06-15T01:53:49Z)
• Trustworthy Medical Segmentation with Uncertainty Estimation [0.7829352305480285]
  This paper introduces a new Bayesian deep learning framework for uncertainty quantification in segmentation neural networks. We evaluate the proposed framework on medical image segmentation data from Magnetic Resonances Imaging and Computed Tomography scans. Our experiments on multiple benchmark datasets demonstrate that the proposed framework is more robust to noise and adversarial attacks as compared to state-of-the-art segmentation models.
  arXiv  Detail & Related papers  (2021-11-10T22:46:05Z)
• Bayesian Uncertainty Estimation of Learned Variational MRI Reconstruction [63.202627467245584]
  We introduce a Bayesian variational framework to quantify the model-immanent (epistemic) uncertainty. We demonstrate that our approach yields competitive results for undersampled MRI reconstruction.
  arXiv  Detail & Related papers  (2021-02-12T18:08:14Z)
• Theoretical Insights Into Multiclass Classification: A High-dimensional Asymptotic View [82.80085730891126]
  We provide the first modernally precise analysis of linear multiclass classification. Our analysis reveals that the classification accuracy is highly distribution-dependent. The insights gained may pave the way for a precise understanding of other classification algorithms.
  arXiv  Detail & Related papers  (2020-11-16T05:17:29Z)
• Uncertainty-Aware Few-Shot Image Classification [118.72423376789062]
  Few-shot image classification learns to recognize new categories from limited labelled data. We propose Uncertainty-Aware Few-Shot framework for image classification.
  arXiv  Detail & Related papers  (2020-10-09T12:26:27Z)
• Semi-supervised Medical Image Classification with Relation-driven Self-ensembling Model [71.80319052891817]
  We present a relation-driven semi-supervised framework for medical image classification. It exploits the unlabeled data by encouraging the prediction consistency of given input under perturbations. Our method outperforms many state-of-the-art semi-supervised learning methods on both single-label and multi-label image classification scenarios.
  arXiv  Detail & Related papers  (2020-05-15T06:57:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.