Unleashing the Power of Prompt-driven Nucleus Instance Segmentation

• URL: http://arxiv.org/abs/2311.15939v4
• Date: Wed, 24 Jan 2024 06:04:29 GMT
• Title: Unleashing the Power of Prompt-driven Nucleus Instance Segmentation
• Authors: Zhongyi Shui and Yunlong Zhang and Kai Yao and Chenglu Zhu and Sunyi Zheng and Jingxiong Li and Honglin Li and Yuxuan Sun and Ruizhe Guo and Lin Yang
• Abstract summary: Segment Anything Model (SAM) has earned huge attention in medical image segmentation. We present a novel prompt-driven framework that consists of a nucleus prompter and SAM for automatic nucleus instance segmentation. Our proposed method sets a new state-of-the-art performance on three challenging benchmarks.
• Score: 12.827503504028629
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Nucleus instance segmentation in histology images is crucial for a broad spectrum of clinical applications. Current dominant algorithms rely on regression of nuclear proxy maps. Distinguishing nucleus instances from the estimated maps requires carefully curated post-processing, which is error-prone and parameter-sensitive. Recently, the Segment Anything Model (SAM) has earned huge attention in medical image segmentation, owing to its impressive generalization ability and promptable property. Nevertheless, its potential on nucleus instance segmentation remains largely underexplored. In this paper, we present a novel prompt-driven framework that consists of a nucleus prompter and SAM for automatic nucleus instance segmentation. Specifically, the prompter learns to generate a unique point prompt for each nucleus while the SAM is fine-tuned to output the corresponding mask for the prompted nucleus. Furthermore, we propose the inclusion of adjacent nuclei as negative prompts to enhance the model's capability to identify overlapping nuclei. Without complicated post-processing, our proposed method sets a new state-of-the-art performance on three challenging benchmarks. Code is available at \url{github.com/windygoo/PromptNucSeg}

Related papers

• NuSegDG: Integration of Heterogeneous Space and Gaussian Kernel for Domain-Generalized Nuclei Segmentation [9.332333405703732]
  We propose a domain-generalizable framework for nuclei image segmentation, abbreviated to NuSegDG. HS-Adapter learns multi-dimensional feature representations of different nuclei domains by injecting a small number of trainable parameters into the image encoder of SAM. GKP-Encoder generates density maps driven by a single point, which guides segmentation predictions by mixing position prompts and semantic prompts.
  arXiv  Detail & Related papers  (2024-08-21T17:19:23Z)
• UN-SAM: Universal Prompt-Free Segmentation for Generalized Nuclei Images [47.59627416801523]
  In digital pathology, precise nuclei segmentation is pivotal yet challenged by the diversity of tissue types, staining protocols, and imaging conditions. We propose the Universal prompt-free SAM framework for Nuclei segmentation (UN-SAM) UN-SAM with exceptional performance surpasses state-of-the-arts in nuclei instance and semantic segmentation, especially the generalization capability in zero-shot scenarios.
  arXiv  Detail & Related papers  (2024-02-26T15:35:18Z)
• Segment Any Cell: A SAM-based Auto-prompting Fine-tuning Framework for Nuclei Segmentation [39.81051783009144]
  Segment Any Cell (SAC) is an innovative framework that enhances SAM for nuclei segmentation. SAC integrates a Low-Rank Adaptation (LoRA) within the attention layer of the Transformer to improve the fine-tuning process. Our contributions include a novel prompt generation strategy, automated adaptability for diverse segmentation tasks, and a versatile framework for semantic segmentation challenges.
  arXiv  Detail & Related papers  (2024-01-24T04:23:17Z)
• Prompt-based Grouping Transformer for Nucleus Detection and Classification [70.55961378096116]
  nuclei detection and classification can produce effective information for disease diagnosis. Most existing methods classify nuclei independently or do not make full use of the semantic similarity between nuclei and their grouping features. We propose a novel end-to-end nuclei detection and classification framework based on a grouping transformer-based classifier.
  arXiv  Detail & Related papers  (2023-10-22T04:50:48Z)
• Enhancing Nucleus Segmentation with HARU-Net: A Hybrid Attention Based Residual U-Blocks Network [9.718765096478371]
  Current methods for nucleus segmentation rely on nuclear morphology or contour-based approaches. We propose a dual-branch network using hybrid attention based residual U-blocks for nucleus instance segmentation. Within the network, we propose a context fusion block (CF-block) that effectively extracts and merges contextual information from the network.
  arXiv  Detail & Related papers  (2023-08-07T08:03:20Z)
• Which Pixel to Annotate: a Label-Efficient Nuclei Segmentation Framework [70.18084425770091]
  Deep neural networks have been widely applied in nuclei instance segmentation of H&E stained pathology images. It is inefficient and unnecessary to label all pixels for a dataset of nuclei images which usually contain similar and redundant patterns. We propose a novel full nuclei segmentation framework that chooses only a few image patches to be annotated, augments the training set from the selected samples, and achieves nuclei segmentation in a semi-supervised manner.
  arXiv  Detail & Related papers  (2022-12-20T14:53:26Z)
• Bend-Net: Bending Loss Regularized Multitask Learning Network for Nuclei Segmentation in Histopathology Images [65.47507533905188]
  We propose a novel multitask learning network with a bending loss regularizer to separate overlapped nuclei accurately. The newly proposed multitask learning architecture enhances the generalization by learning shared representation from three tasks. The proposed bending loss defines high penalties to concave contour points with large curvatures, and applies small penalties to convex contour points with small curvatures.
  arXiv  Detail & Related papers  (2021-09-30T17:29:44Z)
• Instance-aware Self-supervised Learning for Nuclei Segmentation [47.07869311690419]
  We propose a novel self-supervised learning framework to exploit the capacity of convolutional neural networks (CNNs) on the nuclei instance segmentation task. The proposed approach involves two sub-tasks, which enable neural networks to implicitly leverage the prior-knowledge of nuclei size and quantity. Experimental results on the publicly available MoNuSeg dataset show that the proposed self-supervised learning approach can remarkably boost the segmentation accuracy of nuclei instance.
  arXiv  Detail & Related papers  (2020-07-22T03:37:14Z)
• Self-Supervised Nuclei Segmentation in Histopathological Images Using Attention [6.3039500405009665]
  We present a self-supervised approach for segmentation of nuclei for whole slide histopathology images. Our method works on the assumption that the size and texture of nuclei can determine the magnification at which a patch is extracted. Our experiments show that with standard post-processing, our method can outperform other unsupervised nuclei segmentation approaches.
  arXiv  Detail & Related papers  (2020-07-16T14:49:20Z)
• Weakly Supervised Deep Nuclei Segmentation Using Partial Points Annotation in Histopathology Images [51.893494939675314]
  We propose a novel weakly supervised segmentation framework based on partial points annotation. We show that our method can achieve competitive performance compared to the fully supervised counterpart and the state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-07-10T15:41:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.