Impact of individual rater style on deep learning uncertainty in medical imaging segmentation

• URL: http://arxiv.org/abs/2105.02197v1
• Date: Wed, 5 May 2021 17:11:18 GMT
• Title: Impact of individual rater style on deep learning uncertainty in medical imaging segmentation
• Authors: Olivier Vincent, Charley Gros, Julien Cohen-Adad
• Abstract summary: This study quantifies rater style in the form of bias and consistency and explores their impacts when used to train deep learning models. Two multi-rater public datasets were used, consisting of brain multiple sclerosis lesion and spinal cord grey matter segmentation.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While multiple studies have explored the relation between inter-rater variability and deep learning model uncertainty in medical segmentation tasks, little is known about the impact of individual rater style. This study quantifies rater style in the form of bias and consistency and explores their impacts when used to train deep learning models. Two multi-rater public datasets were used, consisting of brain multiple sclerosis lesion and spinal cord grey matter segmentation. On both datasets, results show a correlation ($R^2 = 0.60$ and $0.93$) between rater bias and deep learning uncertainty. The impact of label fusion between raters' annotations on this relationship is also explored, and we show that multi-center consensuses are more effective than single-center consensuses to reduce uncertainty, since rater style is mostly center-specific.

Related papers

• Longitudinal Segmentation of MS Lesions via Temporal Difference Weighting [2.0168790328644697]
  We introduce a novel approach that explicitly incorporates temporal differences between baseline and follow-up scans through a unique architectural inductive bias called Difference Weighting Block. We achieve superior scores in lesion segmentation as well as lesion detection as compared to state-of-the-art longitudinal and single timepoint models across two datasets.
  arXiv  Detail & Related papers  (2024-09-20T11:30:54Z)
• Benchmarking Dependence Measures to Prevent Shortcut Learning in Medical Imaging [7.917858344544847]
  Medical imaging cohorts are often confounded by factors such as acquisition devices, hospital sites, patient backgrounds, and many more. Deep learning models tend to learn spurious correlations instead of causally related features, limiting their generalizability to new and unseen data. This problem can be addressed by minimizing dependence measures between intermediate representations of task-related and non-task-related variables.
  arXiv  Detail & Related papers  (2024-07-26T14:54:16Z)
• Improving Multiple Sclerosis Lesion Segmentation Across Clinical Sites: A Federated Learning Approach with Noise-Resilient Training [75.40980802817349]
  Deep learning models have shown promise for automatically segmenting MS lesions, but the scarcity of accurately annotated data hinders progress in this area. We introduce a Decoupled Hard Label Correction (DHLC) strategy that considers the imbalanced distribution and fuzzy boundaries of MS lesions. We also introduce a Centrally Enhanced Label Correction (CELC) strategy, which leverages the aggregated central model as a correction teacher for all sites.
  arXiv  Detail & Related papers  (2023-08-31T00:36:10Z)
• How inter-rater variability relates to aleatoric and epistemic uncertainty: a case study with deep learning-based paraspinal muscle segmentation [1.9624082208594296]
  We study how inter-rater variability affects the reliability of the resulting deep learning algorithms. Our study reveals the interplay between inter-rater variability and uncertainties, affected by choices of label fusion strategies and DL models.
  arXiv  Detail & Related papers  (2023-08-14T06:40:20Z)
• Pain level and pain-related behaviour classification using GRU-based sparsely-connected RNNs [61.080598804629375]
  People with chronic pain unconsciously adapt specific body movements to protect themselves from injury or additional pain. Because there is no dedicated benchmark database to analyse this correlation, we considered one of the specific circumstances that potentially influence a person's biometrics during daily activities. We proposed a sparsely-connected recurrent neural networks (s-RNNs) ensemble with the gated recurrent unit (GRU) that incorporates multiple autoencoders. We conducted several experiments which indicate that the proposed method outperforms the state-of-the-art approaches in classifying both pain level and pain-related behaviour.
  arXiv  Detail & Related papers  (2022-12-20T12:56:28Z)
• Variability Matters : Evaluating inter-rater variability in histopathology for robust cell detection [3.2873782624127843]
  We present a large-scale study on the variability of cell annotations among 120 board-certified pathologists. We show that increasing the data size at the expense of inter-rater variability does not necessarily lead to better-performing models in cell detection. These findings suggest that the evaluation of the annotators may help tackle the fundamental budget issues in the histopathology domain.
  arXiv  Detail & Related papers  (2022-10-11T06:24:55Z)
• Mine yOur owN Anatomy: Revisiting Medical Image Segmentation with Extremely Limited Labels [54.58539616385138]
  We introduce a novel semi-supervised 2D medical image segmentation framework termed Mine yOur owN Anatomy (MONA) First, prior work argues that every pixel equally matters to the model training; we observe empirically that this alone is unlikely to define meaningful anatomical features. Second, we construct a set of objectives that encourage the model to be capable of decomposing medical images into a collection of anatomical features.
  arXiv  Detail & Related papers  (2022-09-27T15:50:31Z)
• Learning shared neural manifolds from multi-subject FMRI data [13.093635609349874]
  We propose a neural network called MRMD-AEmani that learns a common embedding from multiple subjects in an experiment. We show that our learned common space represents antemporal manifold (where new points not seen during training can be mapped), improves the classification of stimulus features of unseen timepoints. We believe this framework can be used for many downstream applications such as guided brain-computer interface (BCI) training in the future.
  arXiv  Detail & Related papers  (2021-12-22T23:08:39Z)
• Estimating and Improving Fairness with Adversarial Learning [65.99330614802388]
  We propose an adversarial multi-task training strategy to simultaneously mitigate and detect bias in the deep learning-based medical image analysis system. Specifically, we propose to add a discrimination module against bias and a critical module that predicts unfairness within the base classification model. We evaluate our framework on a large-scale public-available skin lesion dataset.
  arXiv  Detail & Related papers  (2021-03-07T03:10:32Z)
• Multiple Sclerosis Lesion Activity Segmentation with Attention-Guided Two-Path CNNs [49.32653090178743]
  convolutional neural networks (CNNs) are studied for lesion activity segmentation from two time points. CNNs are designed and evaluated that combine the information from two points in different ways. It is demonstrated that deep learning-based methods outperform classic approaches.
  arXiv  Detail & Related papers  (2020-08-05T08:49:20Z)
• Semi-supervised Medical Image Classification with Relation-driven Self-ensembling Model [71.80319052891817]
  We present a relation-driven semi-supervised framework for medical image classification. It exploits the unlabeled data by encouraging the prediction consistency of given input under perturbations. Our method outperforms many state-of-the-art semi-supervised learning methods on both single-label and multi-label image classification scenarios.
  arXiv  Detail & Related papers  (2020-05-15T06:57:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.