3D-EffiViTCaps: 3D Efficient Vision Transformer with Capsule for Medical Image Segmentation

• URL: http://arxiv.org/abs/2403.16350v1
• Date: Mon, 25 Mar 2024 00:59:35 GMT
• Title: 3D-EffiViTCaps: 3D Efficient Vision Transformer with Capsule for Medical Image Segmentation
• Authors: Dongwei Gan, Ming Chang, Juan Chen,
• Abstract summary: We propose a U-shaped 3D encoder-decoder network named 3D-EffiViTCaps. Our encoder uses capsule blocks and EfficientViT blocks to jointly capture local and global semantic information. Our decoder employs CNN blocks and EfficientViT blocks to catch ffner details for segmentation.
• Score: 3.3561661617196843
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Medical image segmentation (MIS) aims to finely segment various organs. It requires grasping global information from both parts and the entire image for better segmenting, and clinically there are often certain requirements for segmentation efficiency. Convolutional neural networks (CNNs) have made considerable achievements in MIS. However, they are difficult to fully collect global context information and their pooling layer may cause information loss. Capsule networks, which combine the benefits of CNNs while taking into account additional information such as relative location that CNNs do not, have lately demonstrated some advantages in MIS. Vision Transformer (ViT) employs transformers in visual tasks. Transformer based on attention mechanism has excellent global inductive modeling capabilities and is expected to capture longrange information. Moreover, there have been resent studies on making ViT more lightweight to minimize model complexity and increase efficiency. In this paper, we propose a U-shaped 3D encoder-decoder network named 3D-EffiViTCaps, which combines 3D capsule blocks with 3D EfficientViT blocks for MIS. Our encoder uses capsule blocks and EfficientViT blocks to jointly capture local and global semantic information more effectively and efficiently with less information loss, while the decoder employs CNN blocks and EfficientViT blocks to catch ffner details for segmentation. We conduct experiments on various datasets, including iSeg-2017, Hippocampus and Cardiac to verify the performance and efficiency of 3D-EffiViTCaps, which performs better than previous 3D CNN-based, 3D Capsule-based and 3D Transformer-based models. We further implement a series of ablation experiments on the main blocks. Our code is available at: https://github.com/HidNeuron/3D-EffiViTCaps.

Related papers

• Make Your ViT-based Multi-view 3D Detectors Faster via Token Compression [78.93023152602525]
  Slow inference speed is one of the most crucial concerns for deploying multi-view 3D detectors to tasks with high real-time requirements like autonomous driving. We propose a simple yet effective method called TokenCompression3D (ToC3D) Our method can nearly maintain the performance of recent SOTA with up to 30% inference speedup, and the improvements are consistent after scaling up the ViT and input resolution.
  arXiv  Detail & Related papers  (2024-09-01T06:58:08Z)
• SegFormer3D: an Efficient Transformer for 3D Medical Image Segmentation [0.13654846342364302]
  We present SegFormer3D, a hierarchical Transformer that calculates attention across multiscale volumetric features. SegFormer3D avoids complex decoders and uses an all-MLP decoder to aggregate local and global attention features. We benchmark SegFormer3D against the current SOTA models on three widely used datasets.
  arXiv  Detail & Related papers  (2024-04-15T22:12:05Z)
• RangeViT: Towards Vision Transformers for 3D Semantic Segmentation in Autonomous Driving [80.14669385741202]
  Vision transformers (ViTs) have achieved state-of-the-art results in many image-based benchmarks. ViTs are notoriously hard to train and require a lot of training data to learn powerful representations. We show that our method, called RangeViT, outperforms existing projection-based methods on nuScenes and Semantic KITTI.
  arXiv  Detail & Related papers  (2023-01-24T18:50:48Z)
• MISSU: 3D Medical Image Segmentation via Self-distilling TransUNet [55.16833099336073]
  We propose to self-distill a Transformer-based UNet for medical image segmentation. It simultaneously learns global semantic information and local spatial-detailed features. Our MISSU achieves the best performance over previous state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-06-02T07:38:53Z)
• Dynamic Linear Transformer for 3D Biomedical Image Segmentation [2.440109381823186]
  Transformer-based neural networks have surpassed promising performance on many biomedical image segmentation tasks. Main challenge for 3D transformer-based segmentation methods is the quadratic complexity introduced by the self-attention mechanism. We propose a novel transformer architecture for 3D medical image segmentation using an encoder-decoder style architecture with linear complexity.
  arXiv  Detail & Related papers  (2022-06-01T21:15:01Z)
• 3DConvCaps: 3DUnet with Convolutional Capsule Encoder for Medical Image Segmentation [1.863532786702135]
  We propose a 3D encoder-decoder network with Convolutional Capsule (called 3DConvCaps) to learn lower-level features (short-range attention) with convolutional layers. Our experiments on multiple datasets including iSeg-2017, Hippocampus, and Cardiac demonstrate that our 3D 3DConvCaps network considerably outperforms previous capsule networks and 3D-UNets.
  arXiv  Detail & Related papers  (2022-05-19T03:00:04Z)
• 3D-UCaps: 3D Capsules Unet for Volumetric Image Segmentation [11.312343928772993]
  We propose 3D-UCaps, a 3D voxel-based Capsule network for medical volumetric image segmentation. Our method outperforms previous Capsule networks and 3D-Unets.
  arXiv  Detail & Related papers  (2022-03-16T22:02:37Z)
• CoTr: Efficiently Bridging CNN and Transformer for 3D Medical Image Segmentation [95.51455777713092]
  Convolutional neural networks (CNNs) have been the de facto standard for nowadays 3D medical image segmentation. We propose a novel framework that efficiently bridges a bf Convolutional neural network and a bf Transformer bf (CoTr) for accurate 3D medical image segmentation.
  arXiv  Detail & Related papers  (2021-03-04T13:34:22Z)
• Learning Hybrid Representations for Automatic 3D Vessel Centerline Extraction [57.74609918453932]
  Automatic blood vessel extraction from 3D medical images is crucial for vascular disease diagnoses. Existing methods may suffer from discontinuities of extracted vessels when segmenting such thin tubular structures from 3D images. We argue that preserving the continuity of extracted vessels requires to take into account the global geometry. We propose a hybrid representation learning approach to address this challenge.
  arXiv  Detail & Related papers  (2020-12-14T05:22:49Z)
• Making a Case for 3D Convolutions for Object Segmentation in Videos [16.167397418720483]
  We show that 3D convolutional networks can be effectively applied to dense video prediction tasks such as salient object segmentation. We propose a 3D decoder architecture, that comprises novel 3D Global Convolution layers and 3D Refinement modules. Our approach outperforms existing state-of-the-arts by a large margin on the DAVIS'16 Unsupervised, FBMS and ViSal benchmarks.
  arXiv  Detail & Related papers  (2020-08-26T12:24:23Z)
• CAKES: Channel-wise Automatic KErnel Shrinking for Efficient 3D Networks [87.02416370081123]
  3D Convolution Neural Networks (CNNs) have been widely applied to 3D scene understanding, such as video analysis and volumetric image recognition. We propose Channel-wise Automatic KErnel Shrinking (CAKES), to enable efficient 3D learning by shrinking standard 3D convolutions into a set of economic operations.
  arXiv  Detail & Related papers  (2020-03-28T14:21:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.