Contextual Embedding Learning to Enhance 2D Networks for Volumetric Image Segmentation

• URL: http://arxiv.org/abs/2404.01723v2
• Date: Sat, 18 May 2024 03:54:21 GMT
• Title: Contextual Embedding Learning to Enhance 2D Networks for Volumetric Image Segmentation
• Authors: Zhuoyuan Wang, Dong Sun, Xiangyun Zeng, Ruodai Wu, Yi Wang,
• Abstract summary: 2D convolutional neural networks (CNNs) can hardly exploit the spatial correlation of volumetric data. We propose a contextual embedding learning approach to facilitate 2D CNNs capturing spatial information properly. Our approach leverages the learned embedding and the slice-wisely neighboring matching as a soft cue to guide the network.
• Score: 5.995633685952995
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The segmentation of organs in volumetric medical images plays an important role in computer-aided diagnosis and treatment/surgery planning. Conventional 2D convolutional neural networks (CNNs) can hardly exploit the spatial correlation of volumetric data. Current 3D CNNs have the advantage to extract more powerful volumetric representations but they usually suffer from occupying excessive memory and computation nevertheless. In this study we aim to enhance the 2D networks with contextual information for better volumetric image segmentation. Accordingly, we propose a contextual embedding learning approach to facilitate 2D CNNs capturing spatial information properly. Our approach leverages the learned embedding and the slice-wisely neighboring matching as a soft cue to guide the network. In such a way, the contextual information can be transferred slice-by-slice thus boosting the volumetric representation of the network. Experiments on challenging prostate MRI dataset (PROMISE12) and abdominal CT dataset (CHAOS) show that our contextual embedding learning can effectively leverage the inter-slice context and improve segmentation performance. The proposed approach is a plug-and-play, and memory-efficient solution to enhance the 2D networks for volumetric segmentation. Our code is publicly available at https://github.com/JuliusWang-7/CE_Block.

Related papers

• Self-supervised learning via inter-modal reconstruction and feature projection networks for label-efficient 3D-to-2D segmentation [4.5206601127476445]
  We propose a novel convolutional neural network (CNN) and self-supervised learning (SSL) method for label-efficient 3D-to-2D segmentation. Results on different datasets demonstrate that the proposed CNN significantly improves the state of the art in scenarios with limited labeled data by up to 8% in Dice score.
  arXiv  Detail & Related papers  (2023-07-06T14:16:25Z)
• Dual Multi-scale Mean Teacher Network for Semi-supervised Infection Segmentation in Chest CT Volume for COVID-19 [76.51091445670596]
  Automated detecting lung infections from computed tomography (CT) data plays an important role for combating COVID-19. Most current COVID-19 infection segmentation methods mainly relied on 2D CT images, which lack 3D sequential constraint. Existing 3D CT segmentation methods focus on single-scale representations, which do not achieve the multiple level receptive field sizes on 3D volume.
  arXiv  Detail & Related papers  (2022-11-10T13:11:21Z)
• Two-Stream Graph Convolutional Network for Intra-oral Scanner Image Segmentation [133.02190910009384]
  We propose a two-stream graph convolutional network (i.e., TSGCN) to handle inter-view confusion between different raw attributes. Our TSGCN significantly outperforms state-of-the-art methods in 3D tooth (surface) segmentation.
  arXiv  Detail & Related papers  (2022-04-19T10:41:09Z)
• Point-Unet: A Context-aware Point-based Neural Network for Volumetric Segmentation [18.81644604997336]
  We propose Point-Unet, a novel method that incorporates the efficiency of deep learning with 3D point clouds into volumetric segmentation. Our key idea is to first predict the regions of interest in the volume by learning an attentional probability map. A comprehensive benchmark on different metrics has shown that our context-aware Point-Unet robustly outperforms the SOTA voxel-based networks.
  arXiv  Detail & Related papers  (2022-03-16T22:02:08Z)
• Shape-consistent Generative Adversarial Networks for multi-modal Medical segmentation maps [10.781866671930857]
  We present a segmentation network using synthesised cardiac volumes for extremely limited datasets. Our solution is based on a 3D cross-modality generative adversarial network to share information between modalities. We show that improved segmentation can be achieved on small datasets when using spatial augmentations.
  arXiv  Detail & Related papers  (2022-01-24T13:57:31Z)
• Multi-Slice Dense-Sparse Learning for Efficient Liver and Tumor Segmentation [4.150096314396549]
  Deep convolutional neural network (DCNNs) has obtained tremendous success in 2D and 3D medical image segmentation. We propose a novel dense-sparse training flow from a data perspective, in which, densely adjacent slices and sparsely adjacent slices are extracted as inputs for regularizing DCNNs. We also design a 2.5D light-weight nnU-Net from a network perspective, in which, depthwise separable convolutions are adopted to improve the efficiency.
  arXiv  Detail & Related papers  (2021-08-15T15:29:48Z)
• Residual Moment Loss for Medical Image Segmentation [56.72261489147506]
  Location information is proven to benefit the deep learning models on capturing the manifold structure of target objects. Most existing methods encode the location information in an implicit way, for the network to learn. We propose a novel loss function, namely residual moment (RM) loss, to explicitly embed the location information of segmentation targets.
  arXiv  Detail & Related papers  (2021-06-27T09:31:49Z)
• Learning Hybrid Representations for Automatic 3D Vessel Centerline Extraction [57.74609918453932]
  Automatic blood vessel extraction from 3D medical images is crucial for vascular disease diagnoses. Existing methods may suffer from discontinuities of extracted vessels when segmenting such thin tubular structures from 3D images. We argue that preserving the continuity of extracted vessels requires to take into account the global geometry. We propose a hybrid representation learning approach to address this challenge.
  arXiv  Detail & Related papers  (2020-12-14T05:22:49Z)
• 3D medical image segmentation with labeled and unlabeled data using autoencoders at the example of liver segmentation in CT images [58.720142291102135]
  This work investigates the potential of autoencoder-extracted features to improve segmentation with a convolutional neural network. A convolutional autoencoder was used to extract features from unlabeled data and a multi-scale, fully convolutional CNN was used to perform the target task of 3D liver segmentation in CT images.
  arXiv  Detail & Related papers  (2020-03-17T20:20:43Z)
• Curriculum By Smoothing [52.08553521577014]
  Convolutional Neural Networks (CNNs) have shown impressive performance in computer vision tasks such as image classification, detection, and segmentation. We propose an elegant curriculum based scheme that smoothes the feature embedding of a CNN using anti-aliasing or low-pass filters. As the amount of information in the feature maps increases during training, the network is able to progressively learn better representations of the data.
  arXiv  Detail & Related papers  (2020-03-03T07:27:44Z)
• ACEnet: Anatomical Context-Encoding Network for Neuroanatomy Segmentation [1.7080853582489066]
  2D deep learning methods are favored for their computational efficiency. Existing 2D deep learning methods are not equipped to effectively capture 3D spatial contextual information. We develop an Anatomical Context- Network (ACEnet) to incorporate 3D spatial and anatomical contexts in 2D convolutional neural networks (CNNs) Our method achieves promising performance compared with state-of-the-art alternative methods for brain structure segmentation.
  arXiv  Detail & Related papers  (2020-02-13T20:48:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.