CATS v2: Hybrid encoders for robust medical segmentation

• URL: http://arxiv.org/abs/2308.06377v3
• Date: Wed, 31 Jan 2024 22:57:22 GMT
• Title: CATS v2: Hybrid encoders for robust medical segmentation
• Authors: Hao Li, Han Liu, Dewei Hu, Xing Yao, Jiacheng Wang, Ipek Oguz
• Abstract summary: Convolutional Neural Networks (CNNs) have exhibited strong performance in medical image segmentation tasks. However, due to the limited field of view of convolution kernel, it is hard for CNNs to fully represent global information. We propose CATS v2 with hybrid encoders, which better leverage both local and global information.
• Score: 12.194439938007672
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Convolutional Neural Networks (CNNs) have exhibited strong performance in medical image segmentation tasks by capturing high-level (local) information, such as edges and textures. However, due to the limited field of view of convolution kernel, it is hard for CNNs to fully represent global information. Recently, transformers have shown good performance for medical image segmentation due to their ability to better model long-range dependencies. Nevertheless, transformers struggle to capture high-level spatial features as effectively as CNNs. A good segmentation model should learn a better representation from local and global features to be both precise and semantically accurate. In our previous work, we proposed CATS, which is a U-shaped segmentation network augmented with transformer encoder. In this work, we further extend this model and propose CATS v2 with hybrid encoders. Specifically, hybrid encoders consist of a CNN-based encoder path paralleled to a transformer path with a shifted window, which better leverage both local and global information to produce robust 3D medical image segmentation. We fuse the information from the convolutional encoder and the transformer at the skip connections of different resolutions to form the final segmentation. The proposed method is evaluated on three public challenge datasets: Beyond the Cranial Vault (BTCV), Cross-Modality Domain Adaptation (CrossMoDA) and task 5 of Medical Segmentation Decathlon (MSD-5), to segment abdominal organs, vestibular schwannoma (VS) and prostate, respectively. Compared with the state-of-the-art methods, our approach demonstrates superior performance in terms of higher Dice scores. Our code is publicly available at https://github.com/MedICL-VU/CATS.

Related papers

• ParaTransCNN: Parallelized TransCNN Encoder for Medical Image Segmentation [7.955518153976858]
  We propose an advanced 2D feature extraction method by combining the convolutional neural network and Transformer architectures. Our method is shown with better segmentation accuracy, especially on small organs.
  arXiv  Detail & Related papers  (2024-01-27T05:58:36Z)
• ConvFormer: Plug-and-Play CNN-Style Transformers for Improving Medical Image Segmentation [10.727162449071155]
  We build CNN-style Transformers (ConvFormer) to promote better attention convergence and thus better segmentation performance. In contrast to positional embedding and tokenization, ConvFormer adopts 2D convolution and max-pooling for both position information preservation and feature size reduction.
  arXiv  Detail & Related papers  (2023-09-09T02:18:17Z)
• ConvTransSeg: A Multi-resolution Convolution-Transformer Network for Medical Image Segmentation [14.485482467748113]
  We propose a hybrid encoder-decoder segmentation model (ConvTransSeg) It consists of a multi-layer CNN as the encoder for feature learning and the corresponding multi-level Transformer as the decoder for segmentation prediction. Our method achieves the best performance in terms of Dice coefficient and average symmetric surface distance measures with low model complexity and memory consumption.
  arXiv  Detail & Related papers  (2022-10-13T14:59:23Z)
• Cats: Complementary CNN and Transformer Encoders for Segmentation [13.288195115791758]
  We propose a model with double encoders for 3D biomedical image segmentation. We fuse the information from the convolutional encoder and the transformer, and pass it to the decoder to obtain the results. Compared to the state-of-the-art models with and without transformers on each task, our proposed method obtains higher Dice scores across the board.
  arXiv  Detail & Related papers  (2022-08-24T14:25:11Z)
• HiFormer: Hierarchical Multi-scale Representations Using Transformers for Medical Image Segmentation [3.478921293603811]
  HiFormer is a novel method that efficiently bridges a CNN and a transformer for medical image segmentation. To secure a fine fusion of global and local features, we propose a Double-Level Fusion (DLF) module in the skip connection of the encoder-decoder structure.
  arXiv  Detail & Related papers  (2022-07-18T11:30:06Z)
• MISSU: 3D Medical Image Segmentation via Self-distilling TransUNet [55.16833099336073]
  We propose to self-distill a Transformer-based UNet for medical image segmentation. It simultaneously learns global semantic information and local spatial-detailed features. Our MISSU achieves the best performance over previous state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-06-02T07:38:53Z)
• nnFormer: Interleaved Transformer for Volumetric Segmentation [50.10441845967601]
  We introduce nnFormer, a powerful segmentation model with an interleaved architecture based on empirical combination of self-attention and convolution. nnFormer achieves tremendous improvements over previous transformer-based methods on two commonly used datasets Synapse and ACDC.
  arXiv  Detail & Related papers  (2021-09-07T17:08:24Z)
• Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation [63.46694853953092]
  Swin-Unet is an Unet-like pure Transformer for medical image segmentation. tokenized image patches are fed into the Transformer-based U-shaped decoder-Decoder architecture.
  arXiv  Detail & Related papers  (2021-05-12T09:30:26Z)
• CoTr: Efficiently Bridging CNN and Transformer for 3D Medical Image Segmentation [95.51455777713092]
  Convolutional neural networks (CNNs) have been the de facto standard for nowadays 3D medical image segmentation. We propose a novel framework that efficiently bridges a bf Convolutional neural network and a bf Transformer bf (CoTr) for accurate 3D medical image segmentation.
  arXiv  Detail & Related papers  (2021-03-04T13:34:22Z)
• TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation [78.01570371790669]
  Medical image segmentation is an essential prerequisite for developing healthcare systems. On various medical image segmentation tasks, the u-shaped architecture, also known as U-Net, has become the de-facto standard. We propose TransUNet, which merits both Transformers and U-Net, as a strong alternative for medical image segmentation.
  arXiv  Detail & Related papers  (2021-02-08T16:10:50Z)
• Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers [149.78470371525754]
  We treat semantic segmentation as a sequence-to-sequence prediction task. Specifically, we deploy a pure transformer to encode an image as a sequence of patches. With the global context modeled in every layer of the transformer, this encoder can be combined with a simple decoder to provide a powerful segmentation model, termed SEgmentation TRansformer (SETR) SETR achieves new state of the art on ADE20K (50.28% mIoU), Pascal Context (55.83% mIoU) and competitive results on Cityscapes.
  arXiv  Detail & Related papers  (2020-12-31T18:55:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.