How inter-rater variability relates to aleatoric and epistemic uncertainty: a case study with deep learning-based paraspinal muscle segmentation

• URL: http://arxiv.org/abs/2308.06964v1
• Date: Mon, 14 Aug 2023 06:40:20 GMT
• Title: How inter-rater variability relates to aleatoric and epistemic uncertainty: a case study with deep learning-based paraspinal muscle segmentation
• Authors: Parinaz Roshanzamir, Hassan Rivaz, Joshua Ahn, Hamza Mirza, Neda Naghdi, Meagan Anstruther, Michele C. Batti\'e, Maryse Fortin, and Yiming Xiao
• Abstract summary: We study how inter-rater variability affects the reliability of the resulting deep learning algorithms. Our study reveals the interplay between inter-rater variability and uncertainties, affected by choices of label fusion strategies and DL models.
• Score: 1.9624082208594296
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Recent developments in deep learning (DL) techniques have led to great performance improvement in medical image segmentation tasks, especially with the latest Transformer model and its variants. While labels from fusing multi-rater manual segmentations are often employed as ideal ground truths in DL model training, inter-rater variability due to factors such as training bias, image noise, and extreme anatomical variability can still affect the performance and uncertainty of the resulting algorithms. Knowledge regarding how inter-rater variability affects the reliability of the resulting DL algorithms, a key element in clinical deployment, can help inform better training data construction and DL models, but has not been explored extensively. In this paper, we measure aleatoric and epistemic uncertainties using test-time augmentation (TTA), test-time dropout (TTD), and deep ensemble to explore their relationship with inter-rater variability. Furthermore, we compare UNet and TransUNet to study the impacts of Transformers on model uncertainty with two label fusion strategies. We conduct a case study using multi-class paraspinal muscle segmentation from T2w MRIs. Our study reveals the interplay between inter-rater variability and uncertainties, affected by choices of label fusion strategies and DL models.

Related papers

• MPRE: Multi-perspective Patient Representation Extractor for Disease Prediction [3.914545513460964]
  We propose the Multi-perspective Patient Representation Extractor (MPRE) for disease prediction. Specifically, we propose Frequency Transformation Module (FTM) to extract the trend and variation information of dynamic features. In the 2D Multi-Extraction Network (2D MEN), we form the 2D temporal tensor based on trend and variation. We also propose the First-Order Difference Attention Mechanism (FODAM) to calculate the contributions of differences in adjacent variations to the disease diagnosis.
  arXiv  Detail & Related papers  (2024-01-01T13:52:05Z)
• PULASki: Learning inter-rater variability using statistical distances to improve probabilistic segmentation [36.136619420474766]
  We propose the PULASki for biomedical image segmentation that accurately captures variability in expert annotations. Our approach makes use of an improved loss function based on statistical distances in a conditional variational autoencoder structure. Our method can also be applied to a wide range of multi-label segmentation tasks and is useful for downstream tasks such as hemodynamic modelling.
  arXiv  Detail & Related papers  (2023-12-25T10:31:22Z)
• The effect of data augmentation and 3D-CNN depth on Alzheimer's Disease detection [51.697248252191265]
  This work summarizes and strictly observes best practices regarding data handling, experimental design, and model evaluation. We focus on Alzheimer's Disease (AD) detection, which serves as a paradigmatic example of challenging problem in healthcare. Within this framework, we train predictive 15 models, considering three different data augmentation strategies and five distinct 3D CNN architectures.
  arXiv  Detail & Related papers  (2023-09-13T10:40:41Z)
• Improving Multiple Sclerosis Lesion Segmentation Across Clinical Sites: A Federated Learning Approach with Noise-Resilient Training [75.40980802817349]
  Deep learning models have shown promise for automatically segmenting MS lesions, but the scarcity of accurately annotated data hinders progress in this area. We introduce a Decoupled Hard Label Correction (DHLC) strategy that considers the imbalanced distribution and fuzzy boundaries of MS lesions. We also introduce a Centrally Enhanced Label Correction (CELC) strategy, which leverages the aggregated central model as a correction teacher for all sites.
  arXiv  Detail & Related papers  (2023-08-31T00:36:10Z)
• Cross-Attention is Not Enough: Incongruity-Aware Dynamic Hierarchical Fusion for Multimodal Affect Recognition [69.32305810128994]
  Incongruity between modalities poses a challenge for multimodal fusion, especially in affect recognition. We propose the Hierarchical Crossmodal Transformer with Dynamic Modality Gating (HCT-DMG), a lightweight incongruity-aware model. HCT-DMG: 1) outperforms previous multimodal models with a reduced size of approximately 0.8M parameters; 2) recognizes hard samples where incongruity makes affect recognition difficult; 3) mitigates the incongruity at the latent level in crossmodal attention.
  arXiv  Detail & Related papers  (2023-05-23T01:24:15Z)
• Reliable Joint Segmentation of Retinal Edema Lesions in OCT Images [55.83984261827332]
  In this paper, we propose a novel reliable multi-scale wavelet-enhanced transformer network. We develop a novel segmentation backbone that integrates a wavelet-enhanced feature extractor network and a multi-scale transformer module. Our proposed method achieves better segmentation accuracy with a high degree of reliability as compared to other state-of-the-art segmentation approaches.
  arXiv  Detail & Related papers  (2022-12-01T07:32:56Z)
• Variability Matters : Evaluating inter-rater variability in histopathology for robust cell detection [3.2873782624127843]
  We present a large-scale study on the variability of cell annotations among 120 board-certified pathologists. We show that increasing the data size at the expense of inter-rater variability does not necessarily lead to better-performing models in cell detection. These findings suggest that the evaluation of the annotators may help tackle the fundamental budget issues in the histopathology domain.
  arXiv  Detail & Related papers  (2022-10-11T06:24:55Z)
• Differentiable Agent-based Epidemiology [71.81552021144589]
  We introduce GradABM: a scalable, differentiable design for agent-based modeling that is amenable to gradient-based learning with automatic differentiation. GradABM can quickly simulate million-size populations in few seconds on commodity hardware, integrate with deep neural networks and ingest heterogeneous data sources.
  arXiv  Detail & Related papers  (2022-07-20T07:32:02Z)
• DLTTA: Dynamic Learning Rate for Test-time Adaptation on Cross-domain Medical Images [56.72015587067494]
  We propose a novel dynamic learning rate adjustment method for test-time adaptation, called DLTTA. Our method achieves effective and fast test-time adaptation with consistent performance improvement over current state-of-the-art test-time adaptation methods.
  arXiv  Detail & Related papers  (2022-05-27T02:34:32Z)
• Impact of individual rater style on deep learning uncertainty in medical imaging segmentation [0.0]
  This study quantifies rater style in the form of bias and consistency and explores their impacts when used to train deep learning models. Two multi-rater public datasets were used, consisting of brain multiple sclerosis lesion and spinal cord grey matter segmentation.
  arXiv  Detail & Related papers  (2021-05-05T17:11:18Z)
• Implanting Synthetic Lesions for Improving Liver Lesion Segmentation in CT Exams [0.0]
  We present a method for implanting realistic lesions in CT slices to provide a rich and controllable set of training samples. We conclude that increasing the variability of lesions synthetically in terms of size, density, shape, and position seems to improve the performance of segmentation models for liver lesion segmentation in CT slices.
  arXiv  Detail & Related papers  (2020-08-11T13:23:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.