Multi-rater Prompting for Ambiguous Medical Image Segmentation

• URL: http://arxiv.org/abs/2404.07580v1
• Date: Thu, 11 Apr 2024 09:13:50 GMT
• Title: Multi-rater Prompting for Ambiguous Medical Image Segmentation
• Authors: Jinhong Wang, Yi Cheng, Jintai Chen, Hongxia Xu, Danny Chen, Jian Wu,
• Abstract summary: Multi-rater annotations commonly occur when medical images are independently annotated by multiple experts (raters) We propose a multi-rater prompt-based approach to address these two challenges altogether.
• Score: 12.452584289825849
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Multi-rater annotations commonly occur when medical images are independently annotated by multiple experts (raters). In this paper, we tackle two challenges arisen in multi-rater annotations for medical image segmentation (called ambiguous medical image segmentation): (1) How to train a deep learning model when a group of raters produces a set of diverse but plausible annotations, and (2) how to fine-tune the model efficiently when computation resources are not available for re-training the entire model on a different dataset domain. We propose a multi-rater prompt-based approach to address these two challenges altogether. Specifically, we introduce a series of rater-aware prompts that can be plugged into the U-Net model for uncertainty estimation to handle multi-annotation cases. During the prompt-based fine-tuning process, only 0.3% of learnable parameters are required to be updated comparing to training the entire model. Further, in order to integrate expert consensus and disagreement, we explore different multi-rater incorporation strategies and design a mix-training strategy for comprehensive insight learning. Extensive experiments verify the effectiveness of our new approach for ambiguous medical image segmentation on two public datasets while alleviating the heavy burden of model re-training.

Related papers

• Diversified and Personalized Multi-rater Medical Image Segmentation [43.47142636000329]
  We propose a two-stage framework named D-Persona (first Diversification and then Personalization). In Stage I, we exploit multiple given annotations to train a Probabilistic U-Net model, with a bound-constrained loss to improve the prediction diversity. In Stage II, we design multiple attention-based projection heads to adaptively query the corresponding expert prompts from the shared latent space, and then perform the personalized medical image segmentation.
  arXiv  Detail & Related papers  (2024-03-20T09:00:19Z)
• Tyche: Stochastic In-Context Learning for Medical Image Segmentation [3.7997415514096926]
  Tyche is a model that uses a context set to generate predictions for previously unseen tasks without the need to retrain. We introduce a novel convolution block architecture that enables interactions among predictions. When combined with appropriate model design and loss functions, Tyche can predict a set of plausible diverse segmentation candidates for new or unseen medical images and segmentation tasks without the need to retrain.
  arXiv  Detail & Related papers  (2024-01-24T18:35:55Z)
• Alternate Diverse Teaching for Semi-supervised Medical Image Segmentation [62.021828104757745]
  We propose AD-MT, an alternate diverse teaching approach in a teacher-student framework. It involves a single student model and two non-trainable teacher models that are momentum-updated periodically and randomly in an alternate fashion.
  arXiv  Detail & Related papers  (2023-11-29T02:44:54Z)
• Annotator Consensus Prediction for Medical Image Segmentation with Diffusion Models [70.3497683558609]
  A major challenge in the segmentation of medical images is the large inter- and intra-observer variability in annotations provided by multiple experts. We propose a novel method for multi-expert prediction using diffusion models.
  arXiv  Detail & Related papers  (2023-06-15T10:01:05Z)
• Domain Generalization for Mammographic Image Analysis with Contrastive Learning [62.25104935889111]
  The training of an efficacious deep learning model requires large data with diverse styles and qualities. A novel contrastive learning is developed to equip the deep learning models with better style generalization capability. The proposed method has been evaluated extensively and rigorously with mammograms from various vendor style domains and several public datasets.
  arXiv  Detail & Related papers  (2023-04-20T11:40:21Z)
• Ambiguous Medical Image Segmentation using Diffusion Models [60.378180265885945]
  We introduce a single diffusion model-based approach that produces multiple plausible outputs by learning a distribution over group insights. Our proposed model generates a distribution of segmentation masks by leveraging the inherent sampling process of diffusion. Comprehensive results show that our proposed approach outperforms existing state-of-the-art ambiguous segmentation networks.
  arXiv  Detail & Related papers  (2023-04-10T17:58:22Z)
• Generalized Multi-Task Learning from Substantially Unlabeled Multi-Source Medical Image Data [11.061381376559053]
  MultiMix is a new multi-task learning model that jointly learns disease classification and anatomical segmentation in a semi-supervised manner. Our experiments with varying quantities of multi-source labeled data in the training sets confirm the effectiveness of MultiMix.
  arXiv  Detail & Related papers  (2021-10-25T18:09:19Z)
• Modality Completion via Gaussian Process Prior Variational Autoencoders for Multi-Modal Glioma Segmentation [75.58395328700821]
  We propose a novel model, Multi-modal Gaussian Process Prior Variational Autoencoder (MGP-VAE), to impute one or more missing sub-modalities for a patient scan. MGP-VAE can leverage the Gaussian Process (GP) prior on the Variational Autoencoder (VAE) to utilize the subjects/patients and sub-modalities correlations. We show the applicability of MGP-VAE on brain tumor segmentation where either, two, or three of four sub-modalities may be missing.
  arXiv  Detail & Related papers  (2021-07-07T19:06:34Z)
• D-LEMA: Deep Learning Ensembles from Multiple Annotations -- Application to Skin Lesion Segmentation [14.266037264648533]
  Leveraging a collection of annotators' opinions for an image is an interesting way of estimating a gold standard. We propose an approach to handle annotators' disagreements when training a deep model.
  arXiv  Detail & Related papers  (2020-12-14T01:51:22Z)
• Few-shot Medical Image Segmentation using a Global Correlation Network with Discriminative Embedding [60.89561661441736]
  We propose a novel method for few-shot medical image segmentation. We construct our few-shot image segmentor using a deep convolutional network trained episodically. We enhance discriminability of deep embedding to encourage clustering of the feature domains of the same class.
  arXiv  Detail & Related papers  (2020-12-10T04:01:07Z)
• MultiMix: Sparingly Supervised, Extreme Multitask Learning From Medical Images [13.690075845927606]
  We propose a novel multitask learning model, namely MultiMix, which jointly learns disease classification and anatomical segmentation in a sparingly supervised manner. Our experiments justify the effectiveness of our multitasking model for the classification of pneumonia and segmentation of lungs from chest X-ray images.
  arXiv  Detail & Related papers  (2020-10-28T03:47:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.