Assessing Test-time Variability for Interactive 3D Medical Image Segmentation with Diverse Point Prompts

• URL: http://arxiv.org/abs/2311.07806v1
• Date: Mon, 13 Nov 2023 23:40:24 GMT
• Title: Assessing Test-time Variability for Interactive 3D Medical Image Segmentation with Diverse Point Prompts
• Authors: Hao Li, Han Liu, Dewei Hu, Jiacheng Wang, Ipek Oguz
• Abstract summary: We assess the test-time variability for interactive medical image segmentation with diverse point prompts. Our goal is to identify a straightforward and efficient approach for optimal prompt selection during test-time. We suggest an optimal strategy for prompt selection during test-time, supported by comprehensive results.
• Score: 13.08275555017179
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Interactive segmentation model leverages prompts from users to produce robust segmentation. This advancement is facilitated by prompt engineering, where interactive prompts serve as strong priors during test-time. However, this is an inherently subjective and hard-to-reproduce process. The variability in user expertise and inherently ambiguous boundaries in medical images can lead to inconsistent prompt selections, potentially affecting segmentation accuracy. This issue has not yet been extensively explored for medical imaging. In this paper, we assess the test-time variability for interactive medical image segmentation with diverse point prompts. For a given target region, the point is classified into three sub-regions: boundary, margin, and center. Our goal is to identify a straightforward and efficient approach for optimal prompt selection during test-time based on three considerations: (1) benefits of additional prompts, (2) effects of prompt placement, and (3) strategies for optimal prompt selection. We conduct extensive experiments on the public Medical Segmentation Decathlon dataset for challenging colon tumor segmentation task. We suggest an optimal strategy for prompt selection during test-time, supported by comprehensive results. The code is publicly available at https://github.com/MedICL-VU/variability

Related papers

• SPA: Efficient User-Preference Alignment against Uncertainty in Medical Image Segmentation [8.34233304138989]
  textbfSPA efficiently adapts to diverse test-time preferences with minimal human interaction. It reduces clinician workload in reaching the preferred segmentation. It demonstrates 1) a significant reduction in clinician time and effort compared with existing interactive segmentation approaches.
  arXiv  Detail & Related papers  (2024-11-23T10:27:08Z)
• LIMIS: Towards Language-based Interactive Medical Image Segmentation [58.553786162527686]
  LIMIS is the first purely language-based interactive medical image segmentation model. We adapt Grounded SAM to the medical domain and design a language-based model interaction strategy. We evaluate LIMIS on three publicly available medical datasets in terms of performance and usability.
  arXiv  Detail & Related papers  (2024-10-22T12:13:47Z)
• Curriculum Prompting Foundation Models for Medical Image Segmentation [17.33821260899367]
  Adapting large pre-trained foundation models, e.g., SAM, for medical image segmentation remains a significant challenge. Past works have been heavily reliant on a singular type of prompt for each instance, necessitating manual input of an ideally correct prompt. We propose to utilize prompts of different granularity, which are sourced from original images to provide a broader scope of clinical insights. In response, we have designed a coarse-to-fine mechanism, referred to as curriculum prompting, that progressively integrates prompts of different types.
  arXiv  Detail & Related papers  (2024-09-01T11:00:18Z)
• PFPs: Prompt-guided Flexible Pathological Segmentation for Diverse Potential Outcomes Using Large Vision and Language Models [12.895542069443438]
  We introduce various task prompts through a Large Language Model (LLM) alongside traditional task tokens to enhance segmentation flexibility. Our contribution is in four-fold: (1) we construct a computational-efficient pipeline that uses finetuned language prompts to guide flexible multi-class segmentation; (2) We compare segmentation performance with fixed prompts against free-text; and (3) We design a multi-task kidney pathology segmentation dataset and the corresponding various free-text prompts.
  arXiv  Detail & Related papers  (2024-07-13T18:51:52Z)
• ScribblePrompt: Fast and Flexible Interactive Segmentation for Any Biomedical Image [4.076537350106898]
  We present emphScribblePrompt, a flexible neural network based interactive segmentation tool for biomedical imaging. In a user study with domain experts, ScribblePrompt reduced annotation time by 28% while improving Dice by 15% compared to the next best method. We showcase ScribblePrompt in an interactive demo, provide code, and release a dataset of scribble annotations at https://scribbleprompt.csail.mit.edu.
  arXiv  Detail & Related papers  (2023-12-12T15:57:03Z)
• Multi-task Explainable Skin Lesion Classification [54.76511683427566]
  We propose a few-shot-based approach for skin lesions that generalizes well with few labelled data. The proposed approach comprises a fusion of a segmentation network that acts as an attention module and classification network.
  arXiv  Detail & Related papers  (2023-10-11T05:49:47Z)
• PE-MED: Prompt Enhancement for Interactive Medical Image Segmentation [9.744164910887223]
  We introduce a novel framework equipped with prompt enhancement, called PE-MED, for interactive medical image segmentation. First, we introduce a Self-Loop strategy to generate warm initial segmentation results based on the first prompt. Second, we propose a novel Prompt Attention Learning Module (PALM) to mine useful prompt information in one interaction.
  arXiv  Detail & Related papers  (2023-08-26T03:11:48Z)
• Bayesian Prompt Learning for Image-Language Model Generalization [64.50204877434878]
  We use the regularization ability of Bayesian methods to frame prompt learning as a variational inference problem. Our approach regularizes the prompt space, reduces overfitting to the seen prompts and improves the prompt generalization on unseen prompts. We demonstrate empirically on 15 benchmarks that Bayesian prompt learning provides an appropriate coverage of the prompt space.
  arXiv  Detail & Related papers  (2022-10-05T17:05:56Z)
• MM-TTA: Multi-Modal Test-Time Adaptation for 3D Semantic Segmentation [104.48766162008815]
  We propose and explore a new multi-modal extension of test-time adaptation for 3D semantic segmentation. To design a framework that can take full advantage of multi-modality, each modality provides regularized self-supervisory signals to other modalities. Our regularized pseudo labels produce stable self-learning signals in numerous multi-modal test-time adaptation scenarios.
  arXiv  Detail & Related papers  (2022-04-27T02:28:12Z)
• Semi-Supervised Variational Reasoning for Medical Dialogue Generation [70.838542865384]
  Two key characteristics are relevant for medical dialogue generation: patient states and physician actions. We propose an end-to-end variational reasoning approach to medical dialogue generation. A physician policy network composed of an action-classifier and two reasoning detectors is proposed for augmented reasoning ability.
  arXiv  Detail & Related papers  (2021-05-13T04:14:35Z)
• Robust Medical Instrument Segmentation Challenge 2019 [56.148440125599905]
  Intraoperative tracking of laparoscopic instruments is often a prerequisite for computer and robotic-assisted interventions. Our challenge was based on a surgical data set comprising 10,040 annotated images acquired from a total of 30 surgical procedures. The results confirm the initial hypothesis, namely that algorithm performance degrades with an increasing domain gap.
  arXiv  Detail & Related papers  (2020-03-23T14:35:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.