Segment Any Cell: A SAM-based Auto-prompting Fine-tuning Framework for Nuclei Segmentation

• URL: http://arxiv.org/abs/2401.13220v1
• Date: Wed, 24 Jan 2024 04:23:17 GMT
• Title: Segment Any Cell: A SAM-based Auto-prompting Fine-tuning Framework for Nuclei Segmentation
• Authors: Saiyang Na, Yuzhi Guo, Feng Jiang, Hehuan Ma and Junzhou Huang
• Abstract summary: Segment Any Cell (SAC) is an innovative framework that enhances SAM for nuclei segmentation. SAC integrates a Low-Rank Adaptation (LoRA) within the attention layer of the Transformer to improve the fine-tuning process. Our contributions include a novel prompt generation strategy, automated adaptability for diverse segmentation tasks, and a versatile framework for semantic segmentation challenges.
• Score: 39.81051783009144
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: In the rapidly evolving field of AI research, foundational models like BERT and GPT have significantly advanced language and vision tasks. The advent of pretrain-prompting models such as ChatGPT and Segmentation Anything Model (SAM) has further revolutionized image segmentation. However, their applications in specialized areas, particularly in nuclei segmentation within medical imaging, reveal a key challenge: the generation of high-quality, informative prompts is as crucial as applying state-of-the-art (SOTA) fine-tuning techniques on foundation models. To address this, we introduce Segment Any Cell (SAC), an innovative framework that enhances SAM specifically for nuclei segmentation. SAC integrates a Low-Rank Adaptation (LoRA) within the attention layer of the Transformer to improve the fine-tuning process, outperforming existing SOTA methods. It also introduces an innovative auto-prompt generator that produces effective prompts to guide segmentation, a critical factor in handling the complexities of nuclei segmentation in biomedical imaging. Our extensive experiments demonstrate the superiority of SAC in nuclei segmentation tasks, proving its effectiveness as a tool for pathologists and researchers. Our contributions include a novel prompt generation strategy, automated adaptability for diverse segmentation tasks, the innovative application of Low-Rank Attention Adaptation in SAM, and a versatile framework for semantic segmentation challenges.

Related papers

• ASPS: Augmented Segment Anything Model for Polyp Segmentation [77.25557224490075]
  The Segment Anything Model (SAM) has introduced unprecedented potential for polyp segmentation. SAM's Transformer-based structure prioritizes global and low-frequency information. CFA integrates a trainable CNN encoder branch with a frozen ViT encoder, enabling the integration of domain-specific knowledge.
  arXiv  Detail & Related papers  (2024-06-30T14:55:32Z)
• MedCLIP-SAM: Bridging Text and Image Towards Universal Medical Image Segmentation [2.2585213273821716]
  We propose a novel framework, called MedCLIP-SAM, that combines CLIP and SAM models to generate segmentation of clinical scans. By extensively testing three diverse segmentation tasks and medical image modalities, our proposed framework has demonstrated excellent accuracy.
  arXiv  Detail & Related papers  (2024-03-29T15:59:11Z)
• UN-SAM: Universal Prompt-Free Segmentation for Generalized Nuclei Images [47.59627416801523]
  In digital pathology, precise nuclei segmentation is pivotal yet challenged by the diversity of tissue types, staining protocols, and imaging conditions. We propose the Universal prompt-free SAM framework for Nuclei segmentation (UN-SAM) UN-SAM with exceptional performance surpasses state-of-the-arts in nuclei instance and semantic segmentation, especially the generalization capability in zero-shot scenarios.
  arXiv  Detail & Related papers  (2024-02-26T15:35:18Z)
• Few-Shot Learning for Annotation-Efficient Nucleus Instance Segmentation [50.407071700154674]
  We propose to formulate annotation-efficient nucleus instance segmentation from the perspective of few-shot learning (FSL) Our work was motivated by that, with the prosperity of computational pathology, an increasing number of fully-annotated datasets are publicly accessible. Extensive experiments on a couple of publicly accessible datasets demonstrate that SGFSIS can outperform other annotation-efficient learning baselines.
  arXiv  Detail & Related papers  (2024-02-26T03:49:18Z)
• Unleashing the Power of Prompt-driven Nucleus Instance Segmentation [12.827503504028629]
  Segment Anything Model (SAM) has earned huge attention in medical image segmentation. We present a novel prompt-driven framework that consists of a nucleus prompter and SAM for automatic nucleus instance segmentation. Our proposed method sets a new state-of-the-art performance on three challenging benchmarks.
  arXiv  Detail & Related papers  (2023-11-27T15:46:47Z)
• SurgicalSAM: Efficient Class Promptable Surgical Instrument Segmentation [65.52097667738884]
  We introduce SurgicalSAM, a novel end-to-end efficient-tuning approach for SAM to integrate surgical-specific information with SAM's pre-trained knowledge for improved generalisation. Specifically, we propose a lightweight prototype-based class prompt encoder for tuning, which directly generates prompt embeddings from class prototypes. In addition, to address the low inter-class variance among surgical instrument categories, we propose contrastive prototype learning.
  arXiv  Detail & Related papers  (2023-08-17T02:51:01Z)
• NASDM: Nuclei-Aware Semantic Histopathology Image Generation Using Diffusion Models [3.2996723916635267]
  First-of-its-kind nuclei-aware semantic tissue generation framework (NASDM) NASDM can synthesize realistic tissue samples given a semantic instance mask of up to six different nuclei types. These synthetic images are useful in applications in pathology, validation of models, and supplementation of existing nuclei segmentation datasets.
  arXiv  Detail & Related papers  (2023-03-20T22:16:03Z)
• Domain Adaptive Nuclei Instance Segmentation and Classification via Category-aware Feature Alignment and Pseudo-labelling [65.40672505658213]
  We propose a novel deep neural network, namely Category-Aware feature alignment and Pseudo-Labelling Network (CAPL-Net) for UDA nuclei instance segmentation and classification. Our approach outperforms state-of-the-art UDA methods with a remarkable margin.
  arXiv  Detail & Related papers  (2022-07-04T07:05:06Z)
• PointNu-Net: Keypoint-assisted Convolutional Neural Network for Simultaneous Multi-tissue Histology Nuclei Segmentation and Classification [23.466331358975044]
  We study and design a new method to simultaneously detect, segment, and classify nuclei from Haematoxylin and Eosin stained histopathology data. We demonstrate the superior performance of our proposed approach for nuclei segmentation and classification across 19 different tissue types.
  arXiv  Detail & Related papers  (2021-11-01T08:29:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.