Reliability of deep learning models for anatomical landmark detection: The role of inter-rater variability

• URL: http://arxiv.org/abs/2411.17850v1
• Date: Tue, 26 Nov 2024 20:07:16 GMT
• Title: Reliability of deep learning models for anatomical landmark detection: The role of inter-rater variability
• Authors: Soorena Salari, Hassan Rivaz, Yiming Xiao,
• Abstract summary: We study different annotation-fusion strategies to preserve inter-rater variability in deep learning models for anatomical landmark detection. Our research highlights the crucial connection between inter-rater variability, DL-models performances, and uncertainty.
• Score: 2.64700310378485
• License:
• Abstract: Automated detection of anatomical landmarks plays a crucial role in many diagnostic and surgical applications. Progresses in deep learning (DL) methods have resulted in significant performance enhancement in tasks related to anatomical landmark detection. While current research focuses on accurately localizing these landmarks in medical scans, the importance of inter-rater annotation variability in building DL models is often overlooked. Understanding how inter-rater variability impacts the performance and reliability of the resulting DL algorithms, which are crucial for clinical deployment, can inform the improvement of training data construction and boost DL models' outcomes. In this paper, we conducted a thorough study of different annotation-fusion strategies to preserve inter-rater variability in DL models for anatomical landmark detection, aiming to boost the performance and reliability of the resulting algorithms. Additionally, we explored the characteristics and reliability of four metrics, including a novel Weighted Coordinate Variance metric to quantify landmark detection uncertainty/inter-rater variability. Our research highlights the crucial connection between inter-rater variability, DL-models performances, and uncertainty, revealing how different approaches for multi-rater landmark annotation fusion can influence these factors.

Related papers

• Targeted Cause Discovery with Data-Driven Learning [66.86881771339145]
  We propose a novel machine learning approach for inferring causal variables of a target variable from observations. We employ a neural network trained to identify causality through supervised learning on simulated data. Empirical results demonstrate the effectiveness of our method in identifying causal relationships within large-scale gene regulatory networks.
  arXiv  Detail & Related papers  (2024-08-29T02:21:11Z)
• Deep State-Space Generative Model For Correlated Time-to-Event Predictions [54.3637600983898]
  We propose a deep latent state-space generative model to capture the interactions among different types of correlated clinical events. Our method also uncovers meaningful insights about the latent correlations among mortality and different types of organ failures.
  arXiv  Detail & Related papers  (2024-07-28T02:42:36Z)
• MPRE: Multi-perspective Patient Representation Extractor for Disease Prediction [3.914545513460964]
  We propose the Multi-perspective Patient Representation Extractor (MPRE) for disease prediction. Specifically, we propose Frequency Transformation Module (FTM) to extract the trend and variation information of dynamic features. In the 2D Multi-Extraction Network (2D MEN), we form the 2D temporal tensor based on trend and variation. We also propose the First-Order Difference Attention Mechanism (FODAM) to calculate the contributions of differences in adjacent variations to the disease diagnosis.
  arXiv  Detail & Related papers  (2024-01-01T13:52:05Z)
• Comprehensive framework for evaluation of deep neural networks in detection and quantification of lymphoma from PET/CT images: clinical insights, pitfalls, and observer agreement analyses [0.9958347059366389]
  This study addresses critical gaps in automated lymphoma segmentation from PET/CT images. Deep learning has been applied for lymphoma lesion segmentation, but few studies incorporate out-of-distribution testing. We show that networks perform better on large, intense lesions with higher metabolic activity.
  arXiv  Detail & Related papers  (2023-11-16T06:58:46Z)
• Improving Multiple Sclerosis Lesion Segmentation Across Clinical Sites: A Federated Learning Approach with Noise-Resilient Training [75.40980802817349]
  Deep learning models have shown promise for automatically segmenting MS lesions, but the scarcity of accurately annotated data hinders progress in this area. We introduce a Decoupled Hard Label Correction (DHLC) strategy that considers the imbalanced distribution and fuzzy boundaries of MS lesions. We also introduce a Centrally Enhanced Label Correction (CELC) strategy, which leverages the aggregated central model as a correction teacher for all sites.
  arXiv  Detail & Related papers  (2023-08-31T00:36:10Z)
• Diagnosing Human-object Interaction Detectors [42.283857276076596]
  We introduce a diagnosis toolbox to provide detailed quantitative break-down analysis of HOI detection models. We analyze eight state-of-the-art HOI detection models and provide valuable diagnosis insights to foster future research.
  arXiv  Detail & Related papers  (2023-08-16T17:39:15Z)
• How inter-rater variability relates to aleatoric and epistemic uncertainty: a case study with deep learning-based paraspinal muscle segmentation [1.9624082208594296]
  We study how inter-rater variability affects the reliability of the resulting deep learning algorithms. Our study reveals the interplay between inter-rater variability and uncertainties, affected by choices of label fusion strategies and DL models.
  arXiv  Detail & Related papers  (2023-08-14T06:40:20Z)
• Variability Matters : Evaluating inter-rater variability in histopathology for robust cell detection [3.2873782624127843]
  We present a large-scale study on the variability of cell annotations among 120 board-certified pathologists. We show that increasing the data size at the expense of inter-rater variability does not necessarily lead to better-performing models in cell detection. These findings suggest that the evaluation of the annotators may help tackle the fundamental budget issues in the histopathology domain.
  arXiv  Detail & Related papers  (2022-10-11T06:24:55Z)
• Real-time landmark detection for precise endoscopic submucosal dissection via shape-aware relation network [51.44506007844284]
  We propose a shape-aware relation network for accurate and real-time landmark detection in endoscopic submucosal dissection surgery. We first devise an algorithm to automatically generate relation keypoint heatmaps, which intuitively represent the prior knowledge of spatial relations among landmarks. We then develop two complementary regularization schemes to progressively incorporate the prior knowledge into the training process.
  arXiv  Detail & Related papers  (2021-11-08T07:57:30Z)
• Learning Neural Causal Models with Active Interventions [83.44636110899742]
  We introduce an active intervention-targeting mechanism which enables a quick identification of the underlying causal structure of the data-generating process. Our method significantly reduces the required number of interactions compared with random intervention targeting. We demonstrate superior performance on multiple benchmarks from simulated to real-world data.
  arXiv  Detail & Related papers  (2021-09-06T13:10:37Z)
• Towards Interaction Detection Using Topological Analysis on Neural Networks [55.74562391439507]
  In neural networks, any interacting features must follow a strongly weighted connection to common hidden units. We propose a new measure for quantifying interaction strength, based upon the well-received theory of persistent homology. A Persistence Interaction detection(PID) algorithm is developed to efficiently detect interactions.
  arXiv  Detail & Related papers  (2020-10-25T02:15:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.