LightM-UNet: Mamba Assists in Lightweight UNet for Medical Image
Segmentation
- URL: http://arxiv.org/abs/2403.05246v2
- Date: Mon, 11 Mar 2024 07:14:36 GMT
- Title: LightM-UNet: Mamba Assists in Lightweight UNet for Medical Image
Segmentation
- Authors: Weibin Liao and Yinghao Zhu and Xinyuan Wang and Chengwei Pan and
Yasha Wang and Liantao Ma
- Abstract summary: We introduce the Lightweight Mamba UNet (LightM-UNet) that integrates Mamba and UNet in a lightweight framework.
Specifically, LightM-UNet leverages the Residual Vision Mamba Layer in a pure Mamba fashion to extract deep semantic features and model long-range spatial dependencies.
Experiments conducted on two real-world 2D/3D datasets demonstrate that LightM-UNet surpasses existing state-of-the-art literature.
- Score: 10.563051220050035
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: UNet and its variants have been widely used in medical image segmentation.
However, these models, especially those based on Transformer architectures,
pose challenges due to their large number of parameters and computational
loads, making them unsuitable for mobile health applications. Recently, State
Space Models (SSMs), exemplified by Mamba, have emerged as competitive
alternatives to CNN and Transformer architectures. Building upon this, we
employ Mamba as a lightweight substitute for CNN and Transformer within UNet,
aiming at tackling challenges stemming from computational resource limitations
in real medical settings. To this end, we introduce the Lightweight Mamba UNet
(LightM-UNet) that integrates Mamba and UNet in a lightweight framework.
Specifically, LightM-UNet leverages the Residual Vision Mamba Layer in a pure
Mamba fashion to extract deep semantic features and model long-range spatial
dependencies, with linear computational complexity. Extensive experiments
conducted on two real-world 2D/3D datasets demonstrate that LightM-UNet
surpasses existing state-of-the-art literature. Notably, when compared to the
renowned nnU-Net, LightM-UNet achieves superior segmentation performance while
drastically reducing parameter and computation costs by 116x and 21x,
respectively. This highlights the potential of Mamba in facilitating model
lightweighting. Our code implementation is publicly available at
https://github.com/MrBlankness/LightM-UNet.
Related papers
- MambaVision: A Hybrid Mamba-Transformer Vision Backbone [54.965143338206644]
We propose a novel hybrid Mamba-Transformer backbone, denoted as MambaVision, which is specifically tailored for vision applications.
Our core contribution includes redesigning the Mamba formulation to enhance its capability for efficient modeling of visual features.
We conduct a comprehensive ablation study on the feasibility of integrating Vision Transformers (ViT) with Mamba.
arXiv Detail & Related papers (2024-07-10T23:02:45Z) - MambaVC: Learned Visual Compression with Selective State Spaces [74.29217829932895]
We introduce MambaVC, a simple, strong and efficient compression network based on SSM.
MambaVC develops a visual state space (VSS) block with a 2D selective scanning (2DSS) module as the nonlinear activation function after each downsampling.
On compression benchmark datasets, MambaVC achieves superior rate-distortion performance with lower computational and memory overheads.
arXiv Detail & Related papers (2024-05-24T10:24:30Z) - SPMamba: State-space model is all you need in speech separation [6.590157910988076]
We propose a network architecture for speech separation using a state-space model.
We adopt the TF-GridNet model as the foundational framework and substitute its Transformer component with a bidirectional Mamba module.
Our experimental results reveal an important role in the performance aspects of Mamba-based models.
arXiv Detail & Related papers (2024-04-02T16:04:31Z) - UltraLight VM-UNet: Parallel Vision Mamba Significantly Reduces Parameters for Skin Lesion Segmentation [2.0555786400946134]
State-space models (SSMs) have become a strong competitor to traditional CNNs and Transformers.
We propose an UltraLight Vision Mamba UNet (UltraLight VM-UNet) based on this.
Specifically, we propose a method for processing features in parallel Vision Mamba, named PVM Layer.
arXiv Detail & Related papers (2024-03-29T08:03:42Z) - ReMamber: Referring Image Segmentation with Mamba Twister [51.291487576255435]
ReMamber is a novel RIS architecture that integrates the power of Mamba with a multi-modal Mamba Twister block.
The Mamba Twister explicitly models image-text interaction, and fuses textual and visual features through its unique channel and spatial twisting mechanism.
arXiv Detail & Related papers (2024-03-26T16:27:37Z) - LKM-UNet: Large Kernel Vision Mamba UNet for Medical Image Segmentation [9.862277278217045]
In this paper, we introduce a Large Kernel Vision Mamba U-shape Network, or LKM-UNet, for medical image segmentation.
A distinguishing feature of our LKM-UNet is its utilization of large Mamba kernels, excelling in locally spatial modeling compared to small kernel-based CNNs and Transformers.
Comprehensive experiments demonstrate the feasibility and the effectiveness of using large-size Mamba kernels to achieve large receptive fields.
arXiv Detail & Related papers (2024-03-12T05:34:51Z) - MiM-ISTD: Mamba-in-Mamba for Efficient Infrared Small Target Detection [72.46396769642787]
We develop a nested structure, Mamba-in-Mamba (MiM-ISTD), for efficient infrared small target detection.
MiM-ISTD is $8 times$ faster than the SOTA method and reduces GPU memory usage by 62.2$%$ when testing on $2048 times 2048$ images.
arXiv Detail & Related papers (2024-03-04T15:57:29Z) - PointMamba: A Simple State Space Model for Point Cloud Analysis [65.59944745840866]
We propose PointMamba, transferring the success of Mamba, a recent representative state space model (SSM), from NLP to point cloud analysis tasks.
Unlike traditional Transformers, PointMamba employs a linear complexity algorithm, presenting global modeling capacity while significantly reducing computational costs.
arXiv Detail & Related papers (2024-02-16T14:56:13Z) - Swin-UMamba: Mamba-based UNet with ImageNet-based pretraining [85.08169822181685]
This paper introduces a novel Mamba-based model, Swin-UMamba, designed specifically for medical image segmentation tasks.
Swin-UMamba demonstrates superior performance with a large margin compared to CNNs, ViTs, and latest Mamba-based models.
arXiv Detail & Related papers (2024-02-05T18:58:11Z) - BlackMamba: Mixture of Experts for State-Space Models [10.209192169793772]
State-space models (SSMs) have recently demonstrated competitive performance to transformers at large-scale language modeling benchmarks.
MoE models have shown remarkable performance while significantly reducing the compute and latency costs of inference.
We present BlackMamba, a novel architecture that combines the Mamba SSM with MoE to obtain the benefits of both.
arXiv Detail & Related papers (2024-02-01T07:15:58Z) - SegMamba: Long-range Sequential Modeling Mamba For 3D Medical Image
Segmentation [17.676472608152704]
We introduce SegMamba, a novel 3D medical image textbfSegmentation textbfMamba model.
SegMamba excels in whole volume feature modeling from a state space model standpoint.
Experiments on the BraTS2023 dataset demonstrate the effectiveness and efficiency of our SegMamba.
arXiv Detail & Related papers (2024-01-24T16:17:23Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.