VMambaMorph: a Multi-Modality Deformable Image Registration Framework based on Visual State Space Model with Cross-Scan Module

• URL: http://arxiv.org/abs/2404.05105v2
• Date: Sun, 14 Apr 2024 18:27:41 GMT
• Title: VMambaMorph: a Multi-Modality Deformable Image Registration Framework based on Visual State Space Model with Cross-Scan Module
• Authors: Ziyang Wang, Jian-Qing Zheng, Chao Ma, Tao Guo,
• Abstract summary: This paper introduces an exploration of VMamba with image registration, named VMambaMorph. A novel hybrid VMamba-CNN network is designed specifically for 3D image registration. We validate VMambaMorph using a public benchmark brain MR-CT registration dataset, comparing its performance against current state-of-the-art methods.
• Score: 19.5487294104318
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Image registration, a critical process in medical imaging, involves aligning different sets of medical imaging data into a single unified coordinate system. Deep learning networks, such as the Convolutional Neural Network (CNN)-based VoxelMorph, Vision Transformer (ViT)-based TransMorph, and State Space Model (SSM)-based MambaMorph, have demonstrated effective performance in this domain. The recent Visual State Space Model (VMamba), which incorporates a cross-scan module with SSM, has exhibited promising improvements in modeling global-range dependencies with efficient computational cost in computer vision tasks. This paper hereby introduces an exploration of VMamba with image registration, named VMambaMorph. This novel hybrid VMamba-CNN network is designed specifically for 3D image registration. Utilizing a U-shaped network architecture, VMambaMorph computes the deformation field based on target and source volumes. The VMamba-based block with 2D cross-scan module is redesigned for 3D volumetric feature processing. To overcome the complex motion and structure on multi-modality images, we further propose a fine-tune recursive registration framework. We validate VMambaMorph using a public benchmark brain MR-CT registration dataset, comparing its performance against current state-of-the-art methods. The results indicate that VMambaMorph achieves competitive registration quality. The code for VMambaMorph with all baseline methods is available on GitHub.

Related papers

• GroupMamba: Parameter-Efficient and Accurate Group Visual State Space Model [66.35608254724566]
  State-space models (SSMs) have showcased effective performance in modeling long-range dependencies with subquadratic complexity. However, pure SSM-based models still face challenges related to stability and achieving optimal performance on computer vision tasks. Our paper addresses the challenges of scaling SSM-based models for computer vision, particularly the instability and inefficiency of large model sizes.
  arXiv  Detail & Related papers  (2024-07-18T17:59:58Z)
• MambaVision: A Hybrid Mamba-Transformer Vision Backbone [54.965143338206644]
  We propose a novel hybrid Mamba-Transformer backbone, denoted as MambaVision, which is specifically tailored for vision applications. Our core contribution includes redesigning the Mamba formulation to enhance its capability for efficient modeling of visual features. We conduct a comprehensive ablation study on the feasibility of integrating Vision Transformers (ViT) with Mamba.
  arXiv  Detail & Related papers  (2024-07-10T23:02:45Z)
• Scaling Efficient Masked Autoencoder Learning on Large Remote Sensing Dataset [66.15872913664407]
  This study introduces textbfRS-4M, a large-scale dataset designed to enable highly efficient MIM training on RS images. We propose an efficient MIM method, termed textbfSelectiveMAE, which dynamically encodes and reconstructs a subset of patch tokens selected based on their semantic richness. Experiments show that SelectiveMAE significantly boosts training efficiency by 2.2-2.7 times and enhances the classification, detection, and segmentation performance of the baseline MIM model.
  arXiv  Detail & Related papers  (2024-06-17T15:41:57Z)
• Efficient Visual State Space Model for Image Deblurring [83.57239834238035]
  Convolutional neural networks (CNNs) and Vision Transformers (ViTs) have achieved excellent performance in image restoration. We propose a simple yet effective visual state space model (EVSSM) for image deblurring.
  arXiv  Detail & Related papers  (2024-05-23T09:13:36Z)
• VM-DDPM: Vision Mamba Diffusion for Medical Image Synthesis [0.8111815974227898]
  We propose the Vision Mamba DDPM (VM-DDPM) based on State Space Model (SSM) To our best knowledge, this is the first medical image synthesis model based on the SSM-CNN hybrid architecture. Our experimental evaluation on three datasets of different scales, i.e., ACDC, BraTS2018, and ChestXRay, demonstrate that VM-DDPM achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2024-05-09T10:41:18Z)
• VM-UNET-V2 Rethinking Vision Mamba UNet for Medical Image Segmentation [8.278068663433261]
  We propose Vison Mamba-UNetV2, inspired by Mamba architecture, to capture contextual information in images. VM-UNetV2 exhibits competitive performance in medical image segmentation tasks. We conduct comprehensive experiments on the ISIC17, ISIC18, CVC-300, CVC-ClinicDB, Kvasir CVC-ColonDB and ETIS-LaribPolypDB public datasets.
  arXiv  Detail & Related papers  (2024-03-14T08:12:39Z)
• Mamba-UNet: UNet-Like Pure Visual Mamba for Medical Image Segmentation [21.1787366866505]
  We propose Mamba-UNet, a novel architecture that synergizes the U-Net in medical image segmentation with Mamba's capability. Mamba-UNet adopts a pure Visual Mamba (VMamba)-based encoder-decoder structure, infused with skip connections to preserve spatial information across different scales of the network.
  arXiv  Detail & Related papers  (2024-02-07T18:33:04Z)
• VM-UNet: Vision Mamba UNet for Medical Image Segmentation [3.170171905334503]
  We propose a U-shape architecture model for medical image segmentation, named Vision Mamba UNet (VM-UNet) We conduct comprehensive experiments on the ISIC17, ISIC18, and Synapse datasets, and the results indicate that VM-UNet performs competitively in medical image segmentation tasks.
  arXiv  Detail & Related papers  (2024-02-04T13:37:21Z)
• VMamba: Visual State Space Model [92.83984290020891]
  VMamba is a vision backbone that works in linear time complexity. At the core of VMamba lies a stack of Visual State-Space (VSS) blocks with the 2D Selective Scan (SS2D) module.
  arXiv  Detail & Related papers  (2024-01-18T17:55:39Z)
• Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model [51.10876815815515]
  We propose a new generic vision backbone with bidirectional Mamba blocks (Vim) Vim marks the image sequences with position embeddings and compresses the visual representation with bidirectional state space models. The results demonstrate that Vim is capable of overcoming the computation & memory constraints on performing Transformer-style understanding for high-resolution images.
  arXiv  Detail & Related papers  (2024-01-17T18:56:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.