trAIce3D: A Prompt-Driven Transformer Based U-Net for Semantic Segmentation of Microglial Cells from Large-Scale 3D Microscopy Images

• URL: http://arxiv.org/abs/2507.22635v1
• Date: Wed, 30 Jul 2025 12:54:53 GMT
• Title: trAIce3D: A Prompt-Driven Transformer Based U-Net for Semantic Segmentation of Microglial Cells from Large-Scale 3D Microscopy Images
• Authors: MohammadAmin Alamalhoda, Arsalan Firoozi, Alessandro Venturino, Sandra Siegert,
• Abstract summary: We introduce trAIce3D, a deep-learning architecture designed for precise microglia segmentation.<n>It employs a two-stage approach: first, a 3D U-Net with vision transformers in the encoder detects somas using a sliding-window technique to cover the entire image.<n>It then refines each soma and its branches by using soma coordinates as a prompt and a 3D window around the target cell as input.<n>trained and evaluated on a dataset of 41,230 microglial cells, trAIce3D significantly improves segmentation accuracy and scalable generalization.
• Score: 39.58317527488534
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The shape of a cell contains essential information about its function within the biological system. Segmenting these structures from large-scale 3D microscopy images is challenging, limiting clinical insights especially for microglia, immune-associated cells involved in neurodegenerative diseases. Existing segmentation methods mainly focus on cell bodies, struggle with overlapping structures, perform poorly on noisy images, require hyperparameter tuning for each new dataset, or rely on tedious semi-automated approaches. We introduce trAIce3D, a deep-learning architecture designed for precise microglia segmentation, capturing both somas and branches. It employs a two-stage approach: first, a 3D U-Net with vision transformers in the encoder detects somas using a sliding-window technique to cover the entire image. Then, the same architecture, enhanced with cross-attention blocks in skip connections, refines each soma and its branches by using soma coordinates as a prompt and a 3D window around the target cell as input. Training occurs in two phases: self-supervised Soma Segmentation, followed by prompt-based Branch Segmentation, leveraging pre-trained weights from the first phase. Trained and evaluated on a dataset of 41,230 microglial cells, trAIce3D significantly improves segmentation accuracy and generalization, enabling scalable analysis of complex cellular morphologies. While optimized for microglia, its architecture can extend to other intricate cell types, such as neurons and astrocytes, broadening its impact on neurobiological research.

Related papers

• CISCA and CytoDArk0: a Cell Instance Segmentation and Classification method for histo(patho)logical image Analyses and a new, open, Nissl-stained dataset for brain cytoarchitecture studies [0.19791587637442667]
  We propose a new deep learning framework, CISCA, for automatic cell instance segmentation and classification in histological slices.<n>At the core of CISCA is a network architecture featuring a lightweight U-Net with three heads in the decoder.<n>We evaluate CISCA against other state-of-the-art methods, demonstrating its versatility, robustness, and accuracy in segmenting and classifying cells.
  arXiv  Detail & Related papers  (2024-09-06T10:34:06Z)
• Tracing and segmentation of molecular patterns in 3-dimensional cryo-et/em density maps through algorithmic image processing and deep learning-based techniques [0.0]
  dissertation focuses on developing sophisticated computational techniques for tracing actin filaments. Three novel methodologies have been developed: BundleTrac, for tracing bundle-like actin filaments found in Stereocilium, Spaghetti Tracer, for tracing filaments that move individually with loosely cohesive movements, and Struwwel Tracer, for tracing randomly orientated actin filaments in the actin network. The second component of the dissertation introduces a convolutional neural network (CNN) based segmentation model to determine the location of protein secondary structures, such as helices and beta-sheets, in medium-resolution (5-10 Angstrom) 3-dimensional cryo-electron microscopy
  arXiv  Detail & Related papers  (2024-03-26T00:41:54Z)
• Learning Multimodal Volumetric Features for Large-Scale Neuron Tracing [72.45257414889478]
  We aim to reduce human workload by predicting connectivity between over-segmented neuron pieces. We first construct a dataset, named FlyTracing, that contains millions of pairwise connections of segments expanding the whole fly brain. We propose a novel connectivity-aware contrastive learning method to generate dense volumetric EM image embedding.
  arXiv  Detail & Related papers  (2024-01-05T19:45:12Z)
• Affine-Consistent Transformer for Multi-Class Cell Nuclei Detection [76.11864242047074]
  We propose a novel Affine-Consistent Transformer (AC-Former), which directly yields a sequence of nucleus positions. We introduce an Adaptive Affine Transformer (AAT) module, which can automatically learn the key spatial transformations to warp original images for local network training. Experimental results demonstrate that the proposed method significantly outperforms existing state-of-the-art algorithms on various benchmarks.
  arXiv  Detail & Related papers  (2023-10-22T02:27:02Z)
• 3D Mitochondria Instance Segmentation with Spatio-Temporal Transformers [101.44668514239959]
  We propose a hybrid encoder-decoder framework that efficiently computes spatial and temporal attentions in parallel. We also introduce a semantic clutter-background adversarial loss during training that aids in the region of mitochondria instances from the background.
  arXiv  Detail & Related papers  (2023-03-21T17:58:49Z)
• Dynamic Linear Transformer for 3D Biomedical Image Segmentation [2.440109381823186]
  Transformer-based neural networks have surpassed promising performance on many biomedical image segmentation tasks. Main challenge for 3D transformer-based segmentation methods is the quadratic complexity introduced by the self-attention mechanism. We propose a novel transformer architecture for 3D medical image segmentation using an encoder-decoder style architecture with linear complexity.
  arXiv  Detail & Related papers  (2022-06-01T21:15:01Z)
• Two-Stream Graph Convolutional Network for Intra-oral Scanner Image Segmentation [133.02190910009384]
  We propose a two-stream graph convolutional network (i.e., TSGCN) to handle inter-view confusion between different raw attributes. Our TSGCN significantly outperforms state-of-the-art methods in 3D tooth (surface) segmentation.
  arXiv  Detail & Related papers  (2022-04-19T10:41:09Z)
• Multi-organ Segmentation Network with Adversarial Performance Validator [10.775440368500416]
  This paper introduces an adversarial performance validation network into a 2D-to-3D segmentation framework. The proposed network converts the 2D-coarse result to 3D high-quality segmentation masks in a coarse-to-fine manner, allowing joint optimization to improve segmentation accuracy. Experiments on the NIH pancreas segmentation dataset demonstrate the proposed network achieves state-of-the-art accuracy on small organ segmentation and outperforms the previous best.
  arXiv  Detail & Related papers  (2022-04-16T18:00:29Z)
• Automatic size and pose homogenization with spatial transformer network to improve and accelerate pediatric segmentation [51.916106055115755]
  We propose a new CNN architecture that is pose and scale invariant thanks to the use of Spatial Transformer Network (STN) Our architecture is composed of three sequential modules that are estimated together during training. We test the proposed method in kidney and renal tumor segmentation on abdominal pediatric CT scanners.
  arXiv  Detail & Related papers  (2021-07-06T14:50:03Z)
• RDCNet: Instance segmentation with a minimalist recurrent residual network [0.14999444543328289]
  We propose a minimalist recurrent network called recurrent dilated convolutional network (RDCNet) RDCNet consists of a shared stacked dilated convolution (sSDC) layer that iteratively refines its output and thereby generates interpretable intermediate predictions. We demonstrate its versatility on 3 tasks with different imaging modalities: nuclear segmentation of H&E slides, of 3D anisotropic stacks from light-sheet fluorescence microscopy and leaf segmentation of top-view images of plants.
  arXiv  Detail & Related papers  (2020-10-02T13:36:45Z)
• Cell Segmentation and Tracking using CNN-Based Distance Predictions and a Graph-Based Matching Strategy [0.20999222360659608]
  We present a method for the segmentation of touching cells in microscopy images. By using a novel representation of cell borders, inspired by distance maps, our method is capable to utilize not only touching cells but also close cells in the training process. This representation is notably robust to annotation errors and shows promising results for the segmentation of microscopy images containing in the training data underrepresented or not included cell types.
  arXiv  Detail & Related papers  (2020-04-03T11:55:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.