Progressive Vision-Language Prompt for Multi-Organ Multi-Class Cell Semantic Segmentation with Single Branch

• URL: http://arxiv.org/abs/2412.02978v1
• Date: Wed, 04 Dec 2024 02:44:38 GMT
• Title: Progressive Vision-Language Prompt for Multi-Organ Multi-Class Cell Semantic Segmentation with Single Branch
• Authors: Qing Zhang, Hang Guo, Siyuan Yang, Qingli Li, Yan Wang,
• Abstract summary: Multi-OrgaN multi-Class cell semantic segmentation method with a single brancH (MONCH) Inspired by the synergy of textual and multi-grained visual features, we introduce a progressive prompt decoder to harmonize multimodal information. Experiments on the PanNuke dataset, which has significant class imbalance and subtle cell size and shape variations, demonstrate that MONCH outperforms state-of-the-art cell segmentation methods and vision-language models.
• Score: 23.307707756230513
• License:
• Abstract: Pathological cell semantic segmentation is a fundamental technology in computational pathology, essential for applications like cancer diagnosis and effective treatment. Given that multiple cell types exist across various organs, with subtle differences in cell size and shape, multi-organ, multi-class cell segmentation is particularly challenging. Most existing methods employ multi-branch frameworks to enhance feature extraction, but often result in complex architectures. Moreover, reliance on visual information limits performance in multi-class analysis due to intricate textural details. To address these challenges, we propose a Multi-OrgaN multi-Class cell semantic segmentation method with a single brancH (MONCH) that leverages vision-language input. Specifically, we design a hierarchical feature extraction mechanism to provide coarse-to-fine-grained features for segmenting cells of various shapes, including high-frequency, convolutional, and topological features. Inspired by the synergy of textual and multi-grained visual features, we introduce a progressive prompt decoder to harmonize multimodal information, integrating features from fine to coarse granularity for better context capture. Extensive experiments on the PanNuke dataset, which has significant class imbalance and subtle cell size and shape variations, demonstrate that MONCH outperforms state-of-the-art cell segmentation methods and vision-language models. Codes and implementations will be made publicly available.

Related papers

• CellPilot [3.2096430458509317]
  This work introduces CellPilot, a framework that bridges the gap between automatic and interactive segmentation. Our model was trained on over 675,000 masks of nine diverse cell and gland segmentation datasets, spanning 16 organs. We make the model and a graphical user interface designed to assist practitioners in creating large-scale annotated datasets available as open-source.
  arXiv  Detail & Related papers  (2024-11-23T10:31:10Z)
• PSVMA+: Exploring Multi-granularity Semantic-visual Adaption for Generalized Zero-shot Learning [116.33775552866476]
  Generalized zero-shot learning (GZSL) endeavors to identify the unseen using knowledge from the seen domain. GZSL suffers from insufficient visual-semantic correspondences due to attribute diversity and instance diversity. We propose a multi-granularity progressive semantic-visual adaption network, where sufficient visual elements can be gathered to remedy the inconsistency.
  arXiv  Detail & Related papers  (2024-10-15T12:49:33Z)
• MorphoSeg: An Uncertainty-Aware Deep Learning Method for Biomedical Segmentation of Complex Cellular Morphologies [5.50767638479269]
  Deep learning has revolutionized medical and biological imaging, particularly in segmentation tasks. segmenting biological cells remains challenging due to the high variability and complexity of cell shapes. We introduce a novel benchmark dataset of Ntera-2 cells, a pluripotent carcinoma cell line, exhibiting diverse morphologies across multiple stages of differentiation. We propose an uncertainty-aware deep learning framework for complex cellular morphology segmentation (MorphoSeg) by incorporating sampling of virtual outliers from low-likelihood regions during training.
  arXiv  Detail & Related papers  (2024-09-25T17:25:06Z)
• UniCell: Universal Cell Nucleus Classification via Prompt Learning [76.11864242047074]
  We propose a universal cell nucleus classification framework (UniCell) It employs a novel prompt learning mechanism to uniformly predict the corresponding categories of pathological images from different dataset domains. In particular, our framework adopts an end-to-end architecture for nuclei detection and classification, and utilizes flexible prediction heads for adapting various datasets.
  arXiv  Detail & Related papers  (2024-02-20T11:50:27Z)
• Single-cell Multi-view Clustering via Community Detection with Unknown Number of Clusters [64.31109141089598]
  We introduce scUNC, an innovative multi-view clustering approach tailored for single-cell data. scUNC seamlessly integrates information from different views without the need for a predefined number of clusters. We conducted a comprehensive evaluation of scUNC using three distinct single-cell datasets.
  arXiv  Detail & Related papers  (2023-11-28T08:34:58Z)
• Topology-Guided Multi-Class Cell Context Generation for Digital Pathology [28.43244574309888]
  We introduce several mathematical tools from spatial statistics and topological data analysis. We generate high quality multi-class cell layouts for the first time. We show that the topology-rich cell layouts can be used for data augmentation and improve the performance of downstream tasks such as cell classification.
  arXiv  Detail & Related papers  (2023-04-05T07:01:34Z)
• Learning from partially labeled data for multi-organ and tumor segmentation [102.55303521877933]
  We propose a Transformer based dynamic on-demand network (TransDoDNet) that learns to segment organs and tumors on multiple datasets. A dynamic head enables the network to accomplish multiple segmentation tasks flexibly. We create a large-scale partially labeled Multi-Organ and Tumor benchmark, termed MOTS, and demonstrate the superior performance of our TransDoDNet over other competitors.
  arXiv  Detail & Related papers  (2022-11-13T13:03:09Z)
• Multi-Class Cell Detection Using Spatial Context Representation [23.542218679448624]
  In digital pathology, both detection and classification of cells are important for automatic diagnostic and prognostic tasks. Existing methods focus on morphological appearance of individual cells, whereas in practice pathologists often infer cell classes through their spatial context. We propose a novel method for both detection and classification that explicitly incorporates spatial contextual information.
  arXiv  Detail & Related papers  (2021-10-10T19:54:40Z)
• w-Net: Dual Supervised Medical Image Segmentation Model with Multi-Dimensional Attention and Cascade Multi-Scale Convolution [47.56835064059436]
  Multi-dimensional attention segmentation model with cascade multi-scale convolution is proposed to predict accurate segmentation for small objects in medical images. The proposed method is evaluated on three datasets: KiTS19, Pancreas CT of Decathlon-10, and MICCAI 2018 LiTS Challenge.
  arXiv  Detail & Related papers  (2020-11-15T13:54:22Z)
• Fine-Grained Visual Classification via Progressive Multi-Granularity Training of Jigsaw Patches [67.51747235117]
  Fine-grained visual classification (FGVC) is much more challenging than traditional classification tasks. Recent works mainly tackle this problem by focusing on how to locate the most discriminative parts. We propose a novel framework for fine-grained visual classification to tackle these problems.
  arXiv  Detail & Related papers  (2020-03-08T19:27:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.