CellViT++: Energy-Efficient and Adaptive Cell Segmentation and Classification Using Foundation Models

• URL: http://arxiv.org/abs/2501.05269v1
• Date: Thu, 09 Jan 2025 14:26:50 GMT
• Title: CellViT++: Energy-Efficient and Adaptive Cell Segmentation and Classification Using Foundation Models
• Authors: Fabian Hörst, Moritz Rempe, Helmut Becker, Lukas Heine, Julius Keyl, Jens Kleesiek,
• Abstract summary: $textCellViTscriptstyle ++$ is a framework for generalized cell segmentation in digital pathology. $textCellViTscriptstyle ++$ is an open-source framework featuring a user-friendly, web-based interface for visualization and annotation.
• Score: 1.7674154313605157
• License:
• Abstract: Digital Pathology is a cornerstone in the diagnosis and treatment of diseases. A key task in this field is the identification and segmentation of cells in hematoxylin and eosin-stained images. Existing methods for cell segmentation often require extensive annotated datasets for training and are limited to a predefined cell classification scheme. To overcome these limitations, we propose $\text{CellViT}^{{\scriptscriptstyle ++}}$, a framework for generalized cell segmentation in digital pathology. $\text{CellViT}^{{\scriptscriptstyle ++}}$ utilizes Vision Transformers with foundation models as encoders to compute deep cell features and segmentation masks simultaneously. To adapt to unseen cell types, we rely on a computationally efficient approach. It requires minimal data for training and leads to a drastically reduced carbon footprint. We demonstrate excellent performance on seven different datasets, covering a broad spectrum of cell types, organs, and clinical settings. The framework achieves remarkable zero-shot segmentation and data-efficient cell-type classification. Furthermore, we show that $\text{CellViT}^{{\scriptscriptstyle ++}}$ can leverage immunofluorescence stainings to generate training datasets without the need for pathologist annotations. The automated dataset generation approach surpasses the performance of networks trained on manually labeled data, demonstrating its effectiveness in creating high-quality training datasets without expert annotations. To advance digital pathology, $\text{CellViT}^{{\scriptscriptstyle ++}}$ is available as an open-source framework featuring a user-friendly, web-based interface for visualization and annotation. The code is available under https://github.com/TIO-IKIM/CellViT-plus-plus.

Related papers

• CellPilot [3.2096430458509317]
  This work introduces CellPilot, a framework that bridges the gap between automatic and interactive segmentation. Our model was trained on over 675,000 masks of nine diverse cell and gland segmentation datasets, spanning 16 organs. We make the model and a graphical user interface designed to assist practitioners in creating large-scale annotated datasets available as open-source.
  arXiv  Detail & Related papers  (2024-11-23T10:31:10Z)
• LangCell: Language-Cell Pre-training for Cell Identity Understanding [3.6518971609937068]
  We introduce LangCell, a unified representation of single-cell data and natural language during the pre-training phase. Results show that LangCell is the only single-cell PLM that can work effectively in zero-shot cell identity understanding scenarios.
  arXiv  Detail & Related papers  (2024-05-09T10:04:05Z)
• Cell Graph Transformer for Nuclei Classification [78.47566396839628]
  We develop a cell graph transformer (CGT) that treats nodes and edges as input tokens to enable learnable adjacency and information exchange among all nodes. Poorly features can lead to noisy self-attention scores and inferior convergence. We propose a novel topology-aware pretraining method that leverages a graph convolutional network (GCN) to learn a feature extractor.
  arXiv  Detail & Related papers  (2024-02-20T12:01:30Z)
• UniCell: Universal Cell Nucleus Classification via Prompt Learning [76.11864242047074]
  We propose a universal cell nucleus classification framework (UniCell) It employs a novel prompt learning mechanism to uniformly predict the corresponding categories of pathological images from different dataset domains. In particular, our framework adopts an end-to-end architecture for nuclei detection and classification, and utilizes flexible prediction heads for adapting various datasets.
  arXiv  Detail & Related papers  (2024-02-20T11:50:27Z)
• Diffusion-based Data Augmentation for Nuclei Image Segmentation [68.28350341833526]
  We introduce the first diffusion-based augmentation method for nuclei segmentation. The idea is to synthesize a large number of labeled images to facilitate training the segmentation model. The experimental results show that by augmenting 10% labeled real dataset with synthetic samples, one can achieve comparable segmentation results.
  arXiv  Detail & Related papers  (2023-10-22T06:16:16Z)
• OCELOT: Overlapped Cell on Tissue Dataset for Histopathology [13.691924123273004]
  We release OCELOT, a dataset dedicated to the study of cell-tissue relationships for cell detection in histopathology. We propose multi-task learning approaches that benefit from learning both cell and tissue tasks simultaneously. On the OCELOT test set in particular, we show up to 6.79 improvement in F1-score.
  arXiv  Detail & Related papers  (2023-03-23T08:57:11Z)
• Seamless Iterative Semi-Supervised Correction of Imperfect Labels in Microscopy Images [57.42492501915773]
  In-vitro tests are an alternative to animal testing for the toxicity of medical devices. Human fatigue plays a role in error making, making the use of deep learning appealing. We propose Seamless Iterative Semi-Supervised correction of Imperfect labels (SISSI) Our method successfully provides an adaptive early learning correction technique for object detection.
  arXiv  Detail & Related papers  (2022-08-05T18:52:20Z)
• Comparisons among different stochastic selection of activation layers for convolutional neural networks for healthcare [77.99636165307996]
  We classify biomedical images using ensembles of neural networks. We select our activations among the following ones: ReLU, leaky ReLU, Parametric ReLU, ELU, Adaptive Piecewice Linear Unit, S-Shaped ReLU, Swish, Mish, Mexican Linear Unit, Parametric Deformable Linear Unit, Soft Root Sign.
  arXiv  Detail & Related papers  (2020-11-24T01:53:39Z)
• Towards an Automatic Analysis of CHO-K1 Suspension Growth in Microfluidic Single-cell Cultivation [63.94623495501023]
  We propose a novel Machine Learning architecture, which allows us to infuse a neural deep network with human-powered abstraction on the level of data. Specifically, we train a generative model simultaneously on natural and synthetic data, so that it learns a shared representation, from which a target variable, such as the cell count, can be reliably estimated.
  arXiv  Detail & Related papers  (2020-10-20T08:36:51Z)
• NuClick: A Deep Learning Framework for Interactive Segmentation of Microscopy Images [1.3888122061254422]
  We propose a simple CNN-based approach to speed up collecting annotations for nuclei, cells and glands. For multicellular structures such as glands, we propose a novel approach to provide the NuClick with a squiggle as a guiding signal. We show that NuClick is adaptable to the object scale, robust against variations in the user input, adaptable to new domains, and delivers reliable annotations.
  arXiv  Detail & Related papers  (2020-05-29T11:51:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.