Deep Quality Estimation: Creating Surrogate Models for Human Quality Ratings

• URL: http://arxiv.org/abs/2205.10355v1
• Date: Tue, 17 May 2022 10:32:27 GMT
• Title: Deep Quality Estimation: Creating Surrogate Models for Human Quality Ratings
• Authors: Florian Kofler, Ivan Ezhov, Lucas Fidon, Izabela Horvath, Ezequiel de la Rosa, John LaMaster, Hongwei Li, Tom Finck, Suprosanna Shit, Johannes Paetzold, Spyridon Bakas, Marie Piraud, Jan Kirschke, Tom Vercauteren, Claus Zimmer, Benedikt Wiestler, Bjoern Menze
• Abstract summary: We evaluate on a complex multi-class segmentation problem, specifically glioma segmentation following the BraTS annotation protocol. The training data features quality ratings from 15 expert neuroradiologists on a scale ranging from 1 to 6 stars for various computer-generated and manual 3D annotations. We can approximate segmentation quality within a margin of error comparable to human intra-rater reliability.
• Score: 6.645279583701951
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Human ratings are abstract representations of segmentation quality. To approximate human quality ratings on scarce expert data, we train surrogate quality estimation models. We evaluate on a complex multi-class segmentation problem, specifically glioma segmentation following the BraTS annotation protocol. The training data features quality ratings from 15 expert neuroradiologists on a scale ranging from 1 to 6 stars for various computer-generated and manual 3D annotations. Even though the networks operate on 2D images and with scarce training data, we can approximate segmentation quality within a margin of error comparable to human intra-rater reliability. Segmentation quality prediction has broad applications. While an understanding of segmentation quality is imperative for successful clinical translation of automatic segmentation quality algorithms, it can play an essential role in training new segmentation models. Due to the split-second inference times, it can be directly applied within a loss function or as a fully-automatic dataset curation mechanism in a federated learning setting.

Related papers

• Quality assurance of organs-at-risk delineation in radiotherapy [7.698565355235687]
  The delineation of tumor target and organs-at-risk is critical in the radiotherapy treatment planning. The quality assurance of the automatic segmentation is still an unmet need in clinical practice. Our proposed model, which introduces residual network and attention mechanism in the one-class classification framework, was able to detect the various types of OAR contour errors with high accuracy.
  arXiv  Detail & Related papers  (2024-05-20T02:32:46Z)
• FUSQA: Fetal Ultrasound Segmentation Quality Assessment [1.0819408603463427]
  We propose a simplified Fetal Ultrasound Quality Assessment (FUSQA) model to tackle the segmentation quality assessment. We formulate the segmentation quality assessment process as an automated classification task to distinguish between good and poor-quality segmentation masks for more accurate gestational age estimation.
  arXiv  Detail & Related papers  (2023-03-08T07:45:06Z)
• CONVIQT: Contrastive Video Quality Estimator [63.749184706461826]
  Perceptual video quality assessment (VQA) is an integral component of many streaming and video sharing platforms. Here we consider the problem of learning perceptually relevant video quality representations in a self-supervised manner. Our results indicate that compelling representations with perceptual bearing can be obtained using self-supervised learning.
  arXiv  Detail & Related papers  (2022-06-29T15:22:01Z)
• Cross-Site Severity Assessment of COVID-19 from CT Images via Domain Adaptation [64.59521853145368]
  Early and accurate severity assessment of Coronavirus disease 2019 (COVID-19) based on computed tomography (CT) images offers a great help to the estimation of intensive care unit event. To augment the labeled data and improve the generalization ability of the classification model, it is necessary to aggregate data from multiple sites. This task faces several challenges including class imbalance between mild and severe infections, domain distribution discrepancy between sites, and presence of heterogeneous features.
  arXiv  Detail & Related papers  (2021-09-08T07:56:51Z)
• An Automated Quality Evaluation Framework of Psychotherapy Conversations with Local Quality Estimates [38.841853815519734]
  We propose a hierarchical framework to automatically evaluate the quality of a CBT interaction. We first fine-tune BERT for predicting segment-level (local) quality scores. We then use segment embeddings as lower-level input to a Bidirectional LSTM-based neural network to predict session-level (global) quality estimates.
  arXiv  Detail & Related papers  (2021-06-15T07:18:30Z)
• Cascaded Robust Learning at Imperfect Labels for Chest X-ray Segmentation [61.09321488002978]
  We present a novel cascaded robust learning framework for chest X-ray segmentation with imperfect annotation. Our model consists of three independent network, which can effectively learn useful information from the peer networks. Our methods could achieve a significant improvement on the accuracy in segmentation tasks compared to the previous methods.
  arXiv  Detail & Related papers  (2021-04-05T15:50:16Z)
• Quality-aware semi-supervised learning for CMR segmentation [2.9928692313705505]
  One of the challenges in developing deep learning algorithms for medical image segmentation is the scarcity of training data. We propose a novel scheme that uses QC of the downstream task to identify high quality outputs of CMR segmentation networks. In essence, this provides quality-aware augmentation of training data in a variant of SSL for segmentation networks.
  arXiv  Detail & Related papers  (2020-09-01T17:18:22Z)
• Collaborative Boundary-aware Context Encoding Networks for Error Map Prediction [65.44752447868626]
  We propose collaborative boundaryaware context encoding networks called AEP-Net for error prediction task. Specifically, we propose a collaborative feature transformation branch for better feature fusion between images and masks, and precise localization of error regions. The AEP-Net achieves an average DSC of 0.8358, 0.8164 for error prediction task, and shows a high Pearson correlation coefficient of 0.9873.
  arXiv  Detail & Related papers  (2020-06-25T12:42:01Z)
• Deep Generative Model-based Quality Control for Cardiac MRI Segmentation [30.09405692032434]
  We propose a novel deep generative model-based framework for quality control of cardiac MRI segmentation. The proposed method achieves high prediction accuracy on two publicly available cardiac MRI datasets.
  arXiv  Detail & Related papers  (2020-06-23T23:15:54Z)
• Deep Mining External Imperfect Data for Chest X-ray Disease Screening [57.40329813850719]
  We argue that incorporating an external CXR dataset leads to imperfect training data, which raises the challenges. We formulate the multi-label disease classification problem as weighted independent binary tasks according to the categories. Our framework simultaneously models and tackles the domain and label discrepancies, enabling superior knowledge mining ability.
  arXiv  Detail & Related papers  (2020-06-06T06:48:40Z)
• VerSe: A Vertebrae Labelling and Segmentation Benchmark for Multi-detector CT Images [121.31355003451152]
  Large Scale Vertebrae Challenge (VerSe) was organised in conjunction with the International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI) in 2019 and 2020. We present the the results of this evaluation and further investigate the performance-variation at vertebra-level, scan-level, and at different fields-of-view.
  arXiv  Detail & Related papers  (2020-01-24T21:09:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.