MobileMamba: Lightweight Multi-Receptive Visual Mamba Network

• URL: http://arxiv.org/abs/2411.15941v1
• Date: Sun, 24 Nov 2024 18:01:05 GMT
• Title: MobileMamba: Lightweight Multi-Receptive Visual Mamba Network
• Authors: Haoyang He, Jiangning Zhang, Yuxuan Cai, Hongxu Chen, Xiaobin Hu, Zhenye Gan, Yabiao Wang, Chengjie Wang, Yunsheng Wu, Lei Xie,
• Abstract summary: Previous research on lightweight models has primarily focused on CNNs and Transformer-based designs. We propose the MobileMamba framework, which balances efficiency and performance. MobileMamba achieves up to 83.6% on Top-1, surpassing existing state-of-the-art methods.
• Score: 51.33486891724516
• License:
• Abstract: Previous research on lightweight models has primarily focused on CNNs and Transformer-based designs. CNNs, with their local receptive fields, struggle to capture long-range dependencies, while Transformers, despite their global modeling capabilities, are limited by quadratic computational complexity in high-resolution scenarios. Recently, state-space models have gained popularity in the visual domain due to their linear computational complexity. Despite their low FLOPs, current lightweight Mamba-based models exhibit suboptimal throughput. In this work, we propose the MobileMamba framework, which balances efficiency and performance. We design a three-stage network to enhance inference speed significantly. At a fine-grained level, we introduce the Multi-Receptive Field Feature Interaction(MRFFI) module, comprising the Long-Range Wavelet Transform-Enhanced Mamba(WTE-Mamba), Efficient Multi-Kernel Depthwise Convolution(MK-DeConv), and Eliminate Redundant Identity components. This module integrates multi-receptive field information and enhances high-frequency detail extraction. Additionally, we employ training and testing strategies to further improve performance and efficiency. MobileMamba achieves up to 83.6% on Top-1, surpassing existing state-of-the-art methods which is maximum x21 faster than LocalVim on GPU. Extensive experiments on high-resolution downstream tasks demonstrate that MobileMamba surpasses current efficient models, achieving an optimal balance between speed and accuracy.

Related papers

• MatIR: A Hybrid Mamba-Transformer Image Restoration Model [95.17418386046054]
  We propose a Mamba-Transformer hybrid image restoration model called MatIR. MatIR cross-cycles the blocks of the Transformer layer and the Mamba layer to extract features. In the Mamba module, we introduce the Image Inpainting State Space (IRSS) module, which traverses along four scan paths.
  arXiv  Detail & Related papers  (2025-01-30T14:55:40Z)
• Mamba-SEUNet: Mamba UNet for Monaural Speech Enhancement [54.427965535613886]
  Mamba, as a novel state-space model (SSM), has gained widespread application in natural language processing and computer vision. In this work, we introduce Mamba-SEUNet, an innovative architecture that integrates Mamba with U-Net for SE tasks.
  arXiv  Detail & Related papers  (2024-12-21T13:43:51Z)
• Bi-Mamba: Towards Accurate 1-Bit State Space Models [28.478762133816726]
  Bi-Mamba is a scalable and powerful 1-bit Mamba architecture designed for more efficient large language models. Bi-Mamba achieves performance comparable to its full-precision counterparts (e.g., FP16 or BF16) and much better accuracy than post-training-binarization (PTB) Mamba baselines.
  arXiv  Detail & Related papers  (2024-11-18T18:59:15Z)
• Exploring contextual modeling with linear complexity for point cloud segmentation [43.36716250540622]
  We identify the key components of an effective and efficient point cloud segmentation architecture. We show that Mamba features linear computational complexity, offering superior data and inference efficiency compared to Transformers. We further enhance the standard Mamba specifically for point cloud segmentation by identifying its two key shortcomings.
  arXiv  Detail & Related papers  (2024-10-28T16:56:30Z)
• ReMamba: Equip Mamba with Effective Long-Sequence Modeling [50.530839868893786]
  We propose ReMamba, which enhances Mamba's ability to comprehend long contexts. ReMamba incorporates selective compression and adaptation techniques within a two-stage re-forward process.
  arXiv  Detail & Related papers  (2024-08-28T02:47:27Z)
• LaMamba-Diff: Linear-Time High-Fidelity Diffusion Models Based on Local Attention and Mamba [54.85262314960038]
  Local Attentional Mamba blocks capture both global contexts and local details with linear complexity. Our model exhibits exceptional scalability and surpasses the performance of DiT across various model scales on ImageNet at 256x256 resolution. Compared to state-of-the-art diffusion models on ImageNet 256x256 and 512x512, our largest model presents notable advantages, such as a reduction of up to 62% GFLOPs.
  arXiv  Detail & Related papers  (2024-08-05T16:39:39Z)
• ML-Mamba: Efficient Multi-Modal Large Language Model Utilizing Mamba-2 [4.30176340351235]
  We introduce ML-Mamba, a multimodal language model, which utilizes the latest and efficient Mamba-2 model for inference. We replace the Transformer-based backbone with a pre-trained Mamba-2 model and explore methods for integrating 2D visual selective scanning mechanisms into multimodal learning.
  arXiv  Detail & Related papers  (2024-07-29T09:38:15Z)
• FusionMamba: Dynamic Feature Enhancement for Multimodal Image Fusion with Mamba [19.761723108363796]
  FusionMamba aims to overcome the challenges faced by CNNs and Vision Transformers (ViTs) in computer vision tasks. The framework improves the visual state-space model Mamba by integrating dynamic convolution and channel attention mechanisms. Experiments show that FusionMamba achieves state-of-the-art performance in a variety of multimodal image fusion tasks as well as downstream experiments.
  arXiv  Detail & Related papers  (2024-04-15T06:37:21Z)
• EfficientVMamba: Atrous Selective Scan for Light Weight Visual Mamba [19.062950348441426]
  This work proposes to explore the potential of visual state space models in light-weight model design and introduce a novel efficient model variant dubbed EfficientVMamba. Our EfficientVMamba integrates a atrous-based selective scan approach by efficient skip sampling, constituting building blocks designed to harness both global and local representational features. Experimental results show that, EfficientVMamba scales down the computational complexity while yields competitive results across a variety of vision tasks.
  arXiv  Detail & Related papers  (2024-03-15T02:48:47Z)
• PointMamba: A Simple State Space Model for Point Cloud Analysis [65.59944745840866]
  We propose PointMamba, transferring the success of Mamba, a recent representative state space model (SSM), from NLP to point cloud analysis tasks. Unlike traditional Transformers, PointMamba employs a linear complexity algorithm, presenting global modeling capacity while significantly reducing computational costs.
  arXiv  Detail & Related papers  (2024-02-16T14:56:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.