Rethinking Generalization: The Impact of Annotation Style on Medical Image Segmentation

• URL: http://arxiv.org/abs/2210.17398v1
• Date: Mon, 31 Oct 2022 15:28:49 GMT
• Title: Rethinking Generalization: The Impact of Annotation Style on Medical Image Segmentation
• Authors: Brennan Nichyporuk, Jillian Cardinell, Justin Szeto, Raghav Mehta, Jean-Pierre R. Falet, Douglas L. Arnold, Sotirios A. Tsaftaris, Tal Arbel
• Abstract summary: We show that modeling annotation biases, rather than ignoring them, poses a promising way of accounting for differences in annotation style across datasets. Next, we present an image-conditioning approach to model annotation styles that correlate with specific image features, potentially enabling detection biases to be more easily identified.
• Score: 9.056814157662965
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Generalization is an important attribute of machine learning models, particularly for those that are to be deployed in a medical context, where unreliable predictions can have real world consequences. While the failure of models to generalize across datasets is typically attributed to a mismatch in the data distributions, performance gaps are often a consequence of biases in the ``ground-truth" label annotations. This is particularly important in the context of medical image segmentation of pathological structures (e.g. lesions), where the annotation process is much more subjective, and affected by a number underlying factors, including the annotation protocol, rater education/experience, and clinical aims, among others. In this paper, we show that modeling annotation biases, rather than ignoring them, poses a promising way of accounting for differences in annotation style across datasets. To this end, we propose a generalized conditioning framework to (1) learn and account for different annotation styles across multiple datasets using a single model, (2) identify similar annotation styles across different datasets in order to permit their effective aggregation, and (3) fine-tune a fully trained model to a new annotation style with just a few samples. Next, we present an image-conditioning approach to model annotation styles that correlate with specific image features, potentially enabling detection biases to be more easily identified.

Related papers

• Separating common from salient patterns with Contrastive Representation Learning [2.250968907999846]
  Contrastive Analysis aims at separating common factors of variation between two datasets. Current models based on Variational Auto-Encoders have shown poor performance in learning semantically-expressive representations. We propose to leverage the ability of Contrastive Learning to learn semantically expressive representations well adapted for Contrastive Analysis.
  arXiv  Detail & Related papers  (2024-02-19T08:17:13Z)
• How to Efficiently Annotate Images for Best-Performing Deep Learning Based Segmentation Models: An Empirical Study with Weak and Noisy Annotations and Segment Anything Model [18.293057751504122]
  Deep neural networks (DNNs) have been deployed for many image segmentation tasks and achieved outstanding performance. preparing a dataset for training segmentations is laborious and costly since typically pixel-level annotations are provided for each object of interest. To alleviate this issue, one can provide only weak labels such as bounding boxes or scribbles, or less accurate (noisy) annotations of the objects.
  arXiv  Detail & Related papers  (2023-12-17T04:26:42Z)
• That Label's Got Style: Handling Label Style Bias for Uncertain Image Segmentation [8.363593384698138]
  We show that applying state-of-the-art segmentation uncertainty models on datasets can lead to model bias caused by the different label styles. We present an updated modelling objective conditioning on labeling style for aleatoric uncertainty estimation, and modify two state-of-the-art-architectures for segmentation uncertainty accordingly.
  arXiv  Detail & Related papers  (2023-03-28T09:43:16Z)
• Annotation Error Detection: Analyzing the Past and Present for a More Coherent Future [63.99570204416711]
  We reimplement 18 methods for detecting potential annotation errors and evaluate them on 9 English datasets. We define a uniform evaluation setup including a new formalization of the annotation error detection task. We release our datasets and implementations in an easy-to-use and open source software package.
  arXiv  Detail & Related papers  (2022-06-05T22:31:45Z)
• Resolving label uncertainty with implicit posterior models [71.62113762278963]
  We propose a method for jointly inferring labels across a collection of data samples. By implicitly assuming the existence of a generative model for which a differentiable predictor is the posterior, we derive a training objective that allows learning under weak beliefs.
  arXiv  Detail & Related papers  (2022-02-28T18:09:44Z)
• Disjoint Contrastive Regression Learning for Multi-Sourced Annotations [10.159313152511919]
  Large-scale datasets are important for the development of deep learning models. Multiple annotators may be employed to label different subsets of the data. The inconsistency and bias among different annotators are harmful to the model training.
  arXiv  Detail & Related papers  (2021-12-31T12:39:04Z)
• Learning Debiased and Disentangled Representations for Semantic Segmentation [52.35766945827972]
  We propose a model-agnostic and training scheme for semantic segmentation. By randomly eliminating certain class information in each training iteration, we effectively reduce feature dependencies among classes. Models trained with our approach demonstrate strong results on multiple semantic segmentation benchmarks.
  arXiv  Detail & Related papers  (2021-10-31T16:15:09Z)
• Learning from Partially Overlapping Labels: Image Segmentation under Annotation Shift [68.6874404805223]
  We propose several strategies for learning from partially overlapping labels in the context of abdominal organ segmentation. We find that combining a semi-supervised approach with an adaptive cross entropy loss can successfully exploit heterogeneously annotated data.
  arXiv  Detail & Related papers  (2021-07-13T09:22:24Z)
• A Step Toward More Inclusive People Annotations for Fairness [38.546190750434945]
  We present a new set of annotations on a subset of the Open Images dataset called the MIAP subset. The attributes and labeling methodology for the MIAP subset were designed to enable research into model fairness.
  arXiv  Detail & Related papers  (2021-05-05T20:44:56Z)
• Commonality-Parsing Network across Shape and Appearance for Partially Supervised Instance Segmentation [71.59275788106622]
  We propose to learn the underlying class-agnostic commonalities that can be generalized from mask-annotated categories to novel categories. Our model significantly outperforms the state-of-the-art methods on both partially supervised setting and few-shot setting for instance segmentation on COCO dataset.
  arXiv  Detail & Related papers  (2020-07-24T07:23:44Z)
• Joint Multi-Dimensional Model for Global and Time-Series Annotations [48.159050222769494]
  Crowdsourcing is a popular approach to collect annotations for unlabeled data instances. It involves collecting a large number of annotations from several, often naive untrained annotators for each data instance which are then combined to estimate the ground truth. Most annotation fusion schemes however ignore this aspect and model each dimension separately. We propose a generative model for multi-dimensional annotation fusion, which models the dimensions jointly leading to more accurate ground truth estimates.
  arXiv  Detail & Related papers  (2020-05-06T20:08:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.