Do Public Datasets Assure Unbiased Comparisons for Registration Evaluation?

• URL: http://arxiv.org/abs/2003.09483v1
• Date: Fri, 20 Mar 2020 20:04:47 GMT
• Title: Do Public Datasets Assure Unbiased Comparisons for Registration Evaluation?
• Authors: Jie Luo, Guangshen Ma, Sarah Frisken, Parikshit Juvekar, Nazim Haouchine, Zhe Xu, Yiming Xiao, Alexandra Golby, Patrick Codd, Masashi Sugiyama, and William Wells III
• Abstract summary: We use the variogram to screen the manually annotated landmarks in two datasets used to benchmark registration in image-guided neurosurgeries. Using variograms, we identified potentially problematic cases and had them examined by experienced radiologists.
• Score: 96.53940048041248
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With the increasing availability of new image registration approaches, an unbiased evaluation is becoming more needed so that clinicians can choose the most suitable approaches for their applications. Current evaluations typically use landmarks in manually annotated datasets. As a result, the quality of annotations is crucial for unbiased comparisons. Even though most data providers claim to have quality control over their datasets, an objective third-party screening can be reassuring for intended users. In this study, we use the variogram to screen the manually annotated landmarks in two datasets used to benchmark registration in image-guided neurosurgeries. The variogram provides an intuitive 2D representation of the spatial characteristics of annotated landmarks. Using variograms, we identified potentially problematic cases and had them examined by experienced radiologists. We found that (1) a small number of annotations may have fiducial localization errors; (2) the landmark distribution for some cases is not ideal to offer fair comparisons. If unresolved, both findings could incur bias in registration evaluation.

Related papers

• Mitigating Label Noise on Graph via Topological Sample Selection [72.86862597508077]
  We propose a $textitTopological Sample Selection$ (TSS) method that boosts the informative sample selection process in a graph by utilising topological information. We theoretically prove that our procedure minimizes an upper bound of the expected risk under target clean distribution, and experimentally show the superiority of our method compared with state-of-the-art baselines.
  arXiv  Detail & Related papers  (2024-03-04T11:24:51Z)
• One-Shot Medical Landmark Localization by Edge-Guided Transform and Noisy Landmark Refinement [59.14062241534754]
  We propose a two-stage framework for one-shot medical landmark localization. In stage I, we learn an end-to-end cascade of global alignment and local deformations, under the guidance of novel loss functions. In stage II, we explore self-consistency for selecting reliable pseudo labels and cross-consistency for semi-supervised learning.
  arXiv  Detail & Related papers  (2022-07-31T15:42:28Z)
• Comparison of Evaluation Metrics for Landmark Detection in CMR Images [0.8219153654616499]
  We extend the public ACDC dataset with additional labels of the right ventricular insertion points. We compare different variants of a heatmap-based landmark detection pipeline. Preliminary results indicate that a combination of different metrics is necessary.
  arXiv  Detail & Related papers  (2022-01-25T15:58:30Z)
• Cascaded Robust Learning at Imperfect Labels for Chest X-ray Segmentation [61.09321488002978]
  We present a novel cascaded robust learning framework for chest X-ray segmentation with imperfect annotation. Our model consists of three independent network, which can effectively learn useful information from the peer networks. Our methods could achieve a significant improvement on the accuracy in segmentation tasks compared to the previous methods.
  arXiv  Detail & Related papers  (2021-04-05T15:50:16Z)
• Uncertainty-Aware Annotation Protocol to Evaluate Deformable Registration Algorithms [3.2845753359072125]
  We introduce a principled strategy for the construction of a gold standard in deformable registration. Our framework: (i) iteratively suggests the most informative location to annotate next, taking into account its redundancy with previous annotations; (ii) extends traditional pointwise annotations by accounting for the spatial uncertainty of each annotation; and (iii) naturally provides a new strategy for the evaluation of deformable registration algorithms.
  arXiv  Detail & Related papers  (2021-04-02T19:31:19Z)
• Improving Medical Image Classification with Label Noise Using Dual-uncertainty Estimation [72.0276067144762]
  We discuss and define the two common types of label noise in medical images. We propose an uncertainty estimation-based framework to handle these two label noise amid the medical image classification task.
  arXiv  Detail & Related papers  (2021-02-28T14:56:45Z)
• Assessing the (Un)Trustworthiness of Saliency Maps for Localizing Abnormalities in Medical Imaging [7.359736138662335]
  Saliency maps have become a widely used method to make deep learning models more interpretable. They are increasingly being used in medical imaging to provide clinically plausible explanations. We show that all eight saliency map techniques fail at least one of the criteria and are, in most cases, less trustworthy when compared to the baselines.
  arXiv  Detail & Related papers  (2020-08-06T17:11:19Z)
• Disentangling Human Error from the Ground Truth in Segmentation of Medical Images [12.009437407687987]
  We present a method for jointly learning, from purely noisy observations alone, the reliability of individual annotators and the true segmentation label distributions. We demonstrate the utility of the method on three public medical imaging segmentation datasets with simulated (when necessary) and real diverse annotations.
  arXiv  Detail & Related papers  (2020-07-31T11:03:12Z)
• Semi-supervised Medical Image Classification with Relation-driven Self-ensembling Model [71.80319052891817]
  We present a relation-driven semi-supervised framework for medical image classification. It exploits the unlabeled data by encouraging the prediction consistency of given input under perturbations. Our method outperforms many state-of-the-art semi-supervised learning methods on both single-label and multi-label image classification scenarios.
  arXiv  Detail & Related papers  (2020-05-15T06:57:54Z)
• Semi-supervised lung nodule retrieval [2.055949720959582]
  A content based image retrieval (CBIR) system provides as its output a set of images, ranked by similarity to the query image. Ground truth on similarity between dataset elements (e.g. between nodules) is not readily available, thus greatly challenging machine learning methods. The current study suggests a semi-supervised approach that involves two steps: 1) Automatic annotation of a given partially labeled dataset; 2) Learning a semantic similarity metric space based on the predicated annotations. The proposed system is demonstrated in lung nodule retrieval using the LIDC dataset, and shows that it is feasible to learn embedding from predicted ratings.
  arXiv  Detail & Related papers  (2020-05-04T19:26:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.