Swin SMT: Global Sequential Modeling in 3D Medical Image Segmentation

• URL: http://arxiv.org/abs/2407.07514v1
• Date: Wed, 10 Jul 2024 10:05:22 GMT
• Title: Swin SMT: Global Sequential Modeling in 3D Medical Image Segmentation
• Abstract summary: We introduce Swin Soft Mixture Transformer (Swin SMT), a novel architecture based on Swin UNETR. This model incorporates a Soft Mixture-of-Experts (Soft MoE) to effectively handle complex and diverse long-range dependencies. We evaluate Swin SMT on the publicly available TotalSegmentator-V2 dataset, which includes 117 major anatomical structures in WBCT images.
• Score: 4.915744683251151
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Recent advances in Vision Transformers (ViTs) have significantly enhanced medical image segmentation by facilitating the learning of global relationships. However, these methods face a notable challenge in capturing diverse local and global long-range sequential feature representations, particularly evident in whole-body CT (WBCT) scans. To overcome this limitation, we introduce Swin Soft Mixture Transformer (Swin SMT), a novel architecture based on Swin UNETR. This model incorporates a Soft Mixture-of-Experts (Soft MoE) to effectively handle complex and diverse long-range dependencies. The use of Soft MoE allows for scaling up model parameters maintaining a balance between computational complexity and segmentation performance in both training and inference modes. We evaluate Swin SMT on the publicly available TotalSegmentator-V2 dataset, which includes 117 major anatomical structures in WBCT images. Comprehensive experimental results demonstrate that Swin SMT outperforms several state-of-the-art methods in 3D anatomical structure segmentation, achieving an average Dice Similarity Coefficient of 85.09%. The code and pre-trained weights of Swin SMT are publicly available at https://github.com/MI2DataLab/SwinSMT.

Related papers

• AdaSurvMamba: Dynamic Fusion and Semantic Scanning for Multimodal Survival Analysis [57.942670844512016]
  We introduce AdaSurvMamba as a novel adaptive framework for multimodal survival analysis.<n>The framework features a Dual-Scale Importance-Aware Reconstruction (DSIR) module to dynamically modulate cross-modal interaction strength.<n>Experiments across five TCGA cohorts demonstrate consistent gains over existing methods.
  arXiv  Detail & Related papers  (2026-06-28T11:36:31Z)
• Benchmarking transferability of SSL pretraining to same and different modality segmentation tasks [5.34064424681599]
  Self-distilled masked image transformer (SMIT) achieved the highest overall segmentation accuracy across nine tasks.<n>MIM-based SimMIM and self-distillation methods (DINO, iBOT) outperformed contrastive learning and rotation prediction.
  arXiv  Detail & Related papers  (2026-05-18T14:41:55Z)
• Multimodal Visual Surrogate Compression for Alzheimer's Disease Classification [69.87877580725768]
  Multimodal Visual Surrogate Compression (MVSC) learns to compress and adapt large 3D sMRI volumes into compact 2D features.<n>MVSC has two key components: a Volume Context that captures global cross-slice context under textual guidance, and an Adaptive Slice Fusion module that aggregates slice-level information in a text-enhanced, patch-wise manner.
  arXiv  Detail & Related papers  (2026-01-29T13:05:46Z)
• Mamba Goes HoME: Hierarchical Soft Mixture-of-Experts for 3D Medical Image Segmentation [0.17975553762582286]
  Hierarchical Soft Mixture-of-Experts (HoME) is a token-routing layer for efficient long-context modeling.<n>Built on the Mamba state-space model (SSM) backbone, HoME enhances sequential modeling through sparse, adaptive expert routing.<n>This hierarchical design, combining local expert routing with global expert refinement improves generalizability and segmentation performance.
  arXiv  Detail & Related papers  (2025-07-08T19:57:27Z)
• MRGen: Segmentation Data Engine for Underrepresented MRI Modalities [59.61465292965639]
  Training medical image segmentation models for rare yet clinically important imaging modalities is challenging due to the scarcity of annotated data.<n>This paper investigates leveraging generative models to synthesize data, for training segmentation models for underrepresented modalities.<n>We present MRGen, a data engine for controllable medical image synthesis conditioned on text prompts and segmentation masks.
  arXiv  Detail & Related papers  (2024-12-04T16:34:22Z)
• EM-Net: Efficient Channel and Frequency Learning with Mamba for 3D Medical Image Segmentation [3.6813810514531085]
  We introduce a novel 3D medical image segmentation model called EM-Net. Inspired by its success, we introduce a novel Mamba-based 3D medical image segmentation model called EM-Net. Comprehensive experiments on two challenging multi-organ datasets with other state-of-the-art (SOTA) algorithms show that our method exhibits better segmentation accuracy while requiring nearly half the parameter size of SOTA models and 2x faster training speed.
  arXiv  Detail & Related papers  (2024-09-26T09:34:33Z)
• HMT-UNet: A hybird Mamba-Transformer Vision UNet for Medical Image Segmentation [1.5574423250822542]
  We propose a U-shape architecture model for medical image segmentation, named Hybird Transformer vision Mamba UNet (HTM-UNet) We conduct comprehensive experiments on the ISIC17, ISIC18, CVC-300, CVC-ClinicDB, Kvasir, CVC-ColonDB, ETIS-Larib PolypDB public datasets and ZD-LCI-GIM private dataset.
  arXiv  Detail & Related papers  (2024-08-21T02:25:14Z)
• MM-UNet: A Mixed MLP Architecture for Improved Ophthalmic Image Segmentation [3.2846676620336632]
  Ophthalmic image segmentation serves as a critical foundation for ocular disease diagnosis. Transformer-based models address these limitations but introduce substantial computational overhead. We introduce MM-UNet, an efficient Mixed model tailored for ophthalmic image segmentation.
  arXiv  Detail & Related papers  (2024-08-16T08:34:50Z)
• Rotate to Scan: UNet-like Mamba with Triplet SSM Module for Medical Image Segmentation [8.686237221268584]
  We propose Triplet Mamba-UNet as a new type of image segmentation network. Our model achieves a one-third reduction in parameters compared to the previous VM-UNet.
  arXiv  Detail & Related papers  (2024-03-26T13:40:18Z)
• Enhancing Weakly Supervised 3D Medical Image Segmentation through Probabilistic-aware Learning [52.249748801637196]
  3D medical image segmentation is a challenging task with crucial implications for disease diagnosis and treatment planning. Recent advances in deep learning have significantly enhanced fully supervised medical image segmentation. We propose a novel probabilistic-aware weakly supervised learning pipeline, specifically designed for 3D medical imaging.
  arXiv  Detail & Related papers  (2024-03-05T00:46:53Z)
• SDR-Former: A Siamese Dual-Resolution Transformer for Liver Lesion Classification Using 3D Multi-Phase Imaging [59.78761085714715]
  This study proposes a novel Siamese Dual-Resolution Transformer (SDR-Former) framework for liver lesion classification. The proposed framework has been validated through comprehensive experiments on two clinical datasets. To support the scientific community, we are releasing our extensive multi-phase MR dataset for liver lesion analysis to the public.
  arXiv  Detail & Related papers  (2024-02-27T06:32:56Z)
• 3DSAM-adapter: Holistic adaptation of SAM from 2D to 3D for promptable tumor segmentation [52.699139151447945]
  We propose a novel adaptation method for transferring the segment anything model (SAM) from 2D to 3D for promptable medical image segmentation. Our model can outperform domain state-of-the-art medical image segmentation models on 3 out of 4 tasks, specifically by 8.25%, 29.87%, and 10.11% for kidney tumor, pancreas tumor, colon cancer segmentation, and achieve similar performance for liver tumor segmentation.
  arXiv  Detail & Related papers  (2023-06-23T12:09:52Z)
• AMIGO: Sparse Multi-Modal Graph Transformer with Shared-Context Processing for Representation Learning of Giga-pixel Images [53.29794593104923]
  We present a novel concept of shared-context processing for whole slide histopathology images. AMIGO uses the celluar graph within the tissue to provide a single representation for a patient. We show that our model is strongly robust to missing information to an extent that it can achieve the same performance with as low as 20% of the data.
  arXiv  Detail & Related papers  (2023-03-01T23:37:45Z)
• MISSU: 3D Medical Image Segmentation via Self-distilling TransUNet [55.16833099336073]
  We propose to self-distill a Transformer-based UNet for medical image segmentation. It simultaneously learns global semantic information and local spatial-detailed features. Our MISSU achieves the best performance over previous state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-06-02T07:38:53Z)
• Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images [7.334185314342017]
  We propose a novel segmentation model termed Swin UNEt TRansformers (Swin UNETR) The model extracts features at five different resolutions by utilizing shifted windows for computing self-attention. We have participated in BraTS 2021 segmentation challenge, and our proposed model ranks among the top-performing approaches in the validation phase.
  arXiv  Detail & Related papers  (2022-01-04T18:01:34Z)
• Automatic size and pose homogenization with spatial transformer network to improve and accelerate pediatric segmentation [51.916106055115755]
  We propose a new CNN architecture that is pose and scale invariant thanks to the use of Spatial Transformer Network (STN) Our architecture is composed of three sequential modules that are estimated together during training. We test the proposed method in kidney and renal tumor segmentation on abdominal pediatric CT scanners.
  arXiv  Detail & Related papers  (2021-07-06T14:50:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.