Spectral-Spatial Mamba for Hyperspectral Image Classification

• URL: http://arxiv.org/abs/2404.18401v3
• Date: Thu, 1 Aug 2024 09:04:39 GMT
• Title: Spectral-Spatial Mamba for Hyperspectral Image Classification
• Authors: Lingbo Huang, Yushi Chen, Xin He,
• Abstract summary: spectral-spatial Mamba (SS-Mamba) is applied to hyperspectral image (HSI) classification. The proposed SS-Mamba mainly consists of spectral-spatial token generation module and several stacked spectral-spatial Mamba blocks. The experimental results conducted on widely used HSI datasets reveal that the proposed model achieves competitive results.
• Score: 23.215920983979426
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently, deep learning models have achieved excellent performance in hyperspectral image (HSI) classification. Among the many deep models, Transformer has gradually attracted interest for its excellence in modeling the long-range dependencies of spatial-spectral features in HSI. However, Transformer has the problem of quadratic computational complexity due to the self-attention mechanism, which is heavier than other models and thus has limited adoption in HSI processing. Fortunately, the recently emerging state space model-based Mamba shows great computational efficiency while achieving the modeling power of Transformers. Therefore, in this paper, we make a preliminary attempt to apply the Mamba to HSI classification, leading to the proposed spectral-spatial Mamba (SS-Mamba). Specifically, the proposed SS-Mamba mainly consists of spectral-spatial token generation module and several stacked spectral-spatial Mamba blocks. Firstly, the token generation module converts any given HSI cube to spatial and spectral tokens as sequences. And then these tokens are sent to stacked spectral-spatial mamba blocks (SS-MB). Each SS-MB block consists of two basic mamba blocks and a spectral-spatial feature enhancement module. The spatial and spectral tokens are processed separately by the two basic mamba blocks, respectively. Besides, the feature enhancement module modulates spatial and spectral tokens using HSI sample's center region information. In this way, the spectral and spatial tokens cooperate with each other and achieve information fusion within each block. The experimental results conducted on widely used HSI datasets reveal that the proposed model achieves competitive results compared with the state-of-the-art methods. The Mamba-based method opens a new window for HSI classification.

Related papers

• MambaHSI: Spatial-Spectral Mamba for Hyperspectral Image Classification [46.111607032455225]
  We propose a novel HSI classification model based on a Mamba model, named MambaHSI. Specifically, we design a spatial Mamba block (SpaMB) to model the long-range interaction of the whole image at the pixel-level. We propose a spectral Mamba block (SpeMB) to split the spectral vector into multiple groups, mine the relations across different spectral groups, and extract spectral features.
  arXiv  Detail & Related papers  (2025-01-09T03:27:47Z)
• STNMamba: Mamba-based Spatial-Temporal Normality Learning for Video Anomaly Detection [48.997518615379995]
  Video anomaly detection (VAD) has been extensively researched due to its potential for intelligent video systems. Most existing methods based on CNNs and transformers still suffer from substantial computational burdens. We propose a lightweight and effective Mamba-based network named STNMamba to enhance the learning of spatial-temporal normality.
  arXiv  Detail & Related papers  (2024-12-28T08:49:23Z)
• Spatial and Spatial-Spectral Morphological Mamba for Hyperspectral Image Classification [27.943537708598306]
  We propose the morphological spatial mamba (SMM) and morphological spatial-spectral Mamba (SSMM) model (MorpMamba) MorpMamba combines the strengths of morphological operations and the state space model framework, offering a more computationally efficient alternative to transformers. Experimental results on widely used HSI datasets demonstrate that MorpMamba achieves superior parametric efficiency compared to traditional CNN and transformer models.
  arXiv  Detail & Related papers  (2024-08-02T16:28:51Z)
• GroupMamba: Parameter-Efficient and Accurate Group Visual State Space Model [66.35608254724566]
  State-space models (SSMs) have showcased effective performance in modeling long-range dependencies with subquadratic complexity. However, pure SSM-based models still face challenges related to stability and achieving optimal performance on computer vision tasks. Our paper addresses the challenges of scaling SSM-based models for computer vision, particularly the instability and inefficiency of large model sizes.
  arXiv  Detail & Related papers  (2024-07-18T17:59:58Z)
• DualMamba: A Lightweight Spectral-Spatial Mamba-Convolution Network for Hyperspectral Image Classification [10.329381824237434]
  We propose a novel lightweight parallel design called lightweight dual-stream Mamba-convolution network (DualMamba) for HSI classification. Specifically, a parallel lightweight Mamba and CNN block are first developed to extract global and local spectral-spatial features. Compared with state-of-the-art HSI classification methods, experimental results demonstrate that DualMamba achieves significant classification accuracy.
  arXiv  Detail & Related papers  (2024-06-11T08:26:42Z)
• 3DSS-Mamba: 3D-Spectral-Spatial Mamba for Hyperspectral Image Classification [14.341510793163138]
  We propose a novel 3D-Spectral-Spatial Mamba framework for HSI classification. A 3D-Spectral-Spatial Selective Scanning mechanism is introduced, which performs pixel-wise selective scanning on 3D hyperspectral tokens. Experimental results and analysis demonstrate that the proposed method outperforms the state-of-the-art methods on HSI classification benchmarks.
  arXiv  Detail & Related papers  (2024-05-21T04:10:26Z)
• Mamba-in-Mamba: Centralized Mamba-Cross-Scan in Tokenized Mamba Model for Hyperspectral Image Classification [4.389334324926174]
  This study introduces the innovative Mamba-in-Mamba (MiM) architecture for HSI classification, the first attempt of deploying State Space Model (SSM) in this task. MiM model includes 1) A novel centralized Mamba-Cross-Scan (MCS) mechanism for transforming images into sequence-data, 2) A Tokenized Mamba (T-Mamba) encoder, and 3) A Weighted MCS Fusion (WMF) module. Experimental results from three public HSI datasets demonstrate that our method outperforms existing baselines and state-of-the-art approaches.
  arXiv  Detail & Related papers  (2024-05-20T13:19:02Z)
• Spectral Enhanced Rectangle Transformer for Hyperspectral Image Denoising [64.11157141177208]
  We propose a spectral enhanced rectangle Transformer to model the spatial and spectral correlation in hyperspectral images. For the former, we exploit the rectangle self-attention horizontally and vertically to capture the non-local similarity in the spatial domain. For the latter, we design a spectral enhancement module that is capable of extracting global underlying low-rank property of spatial-spectral cubes to suppress noise.
  arXiv  Detail & Related papers  (2023-04-03T09:42:13Z)
• Coarse-to-Fine Sparse Transformer for Hyperspectral Image Reconstruction [138.04956118993934]
  We propose a novel Transformer-based method, coarse-to-fine sparse Transformer (CST) CST embedding HSI sparsity into deep learning for HSI reconstruction. In particular, CST uses our proposed spectra-aware screening mechanism (SASM) for coarse patch selecting. Then the selected patches are fed into our customized spectra-aggregation hashing multi-head self-attention (SAH-MSA) for fine pixel clustering and self-similarity capturing.
  arXiv  Detail & Related papers  (2022-03-09T16:17:47Z)
• Mask-guided Spectral-wise Transformer for Efficient Hyperspectral Image Reconstruction [127.20208645280438]
  Hyperspectral image (HSI) reconstruction aims to recover the 3D spatial-spectral signal from a 2D measurement. Modeling the inter-spectra interactions is beneficial for HSI reconstruction. Mask-guided Spectral-wise Transformer (MST) proposes a novel framework for HSI reconstruction.
  arXiv  Detail & Related papers  (2021-11-15T16:59:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.