S$^2$Mamba: A Spatial-spectral State Space Model for Hyperspectral Image Classification
- URL: http://arxiv.org/abs/2404.18213v2
- Date: Tue, 13 Aug 2024 10:47:13 GMT
- Title: S$^2$Mamba: A Spatial-spectral State Space Model for Hyperspectral Image Classification
- Authors: Guanchun Wang, Xiangrong Zhang, Zelin Peng, Tianyang Zhang, Licheng Jiao,
- Abstract summary: Land cover analysis using hyperspectral images (HSI) remains an open problem due to their low spatial resolution and complex spectral information.
We propose S$2$Mamba, a spatial-spectral state space model for hyperspectral image classification, to excavate spatial-spectral contextual features.
- Score: 44.99672241508994
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Land cover analysis using hyperspectral images (HSI) remains an open problem due to their low spatial resolution and complex spectral information. Recent studies are primarily dedicated to designing Transformer-based architectures for spatial-spectral long-range dependencies modeling, which is computationally expensive with quadratic complexity. Selective structured state space model (Mamba), which is efficient for modeling long-range dependencies with linear complexity, has recently shown promising progress. However, its potential in hyperspectral image processing that requires handling numerous spectral bands has not yet been explored. In this paper, we innovatively propose S$^2$Mamba, a spatial-spectral state space model for hyperspectral image classification, to excavate spatial-spectral contextual features, resulting in more efficient and accurate land cover analysis. In S$^2$Mamba, two selective structured state space models through different dimensions are designed for feature extraction, one for spatial, and the other for spectral, along with a spatial-spectral mixture gate for optimal fusion. More specifically, S$^2$Mamba first captures spatial contextual relations by interacting each pixel with its adjacent through a Patch Cross Scanning module and then explores semantic information from continuous spectral bands through a Bi-directional Spectral Scanning module. Considering the distinct expertise of the two attributes in homogenous and complicated texture scenes, we realize the Spatial-spectral Mixture Gate by a group of learnable matrices, allowing for the adaptive incorporation of representations learned across different dimensions. Extensive experiments conducted on HSI classification benchmarks demonstrate the superiority and prospect of S$^2$Mamba. The code will be made available at: https://github.com/PURE-melo/S2Mamba.
Related papers
- Spatial-Mamba: Effective Visual State Space Models via Structure-Aware State Fusion [46.82975707531064]
Selective state space models (SSMs) excel at capturing long-range dependencies in 1D sequential data.
We propose Spatial-Mamba, a novel approach that establishes neighborhood connectivity directly in the state space.
We show that Spatial-Mamba, even with a single scan, attains or surpasses the state-of-the-art SSM-based models in image classification, detection and segmentation.
arXiv Detail & Related papers (2024-10-19T12:56:58Z) - Empowering Snapshot Compressive Imaging: Spatial-Spectral State Space Model with Across-Scanning and Local Enhancement [51.557804095896174]
We introduce a State Space Model with Across-Scanning and Local Enhancement, named ASLE-SSM, that employs a Spatial-Spectral SSM for global-local balanced context encoding and cross-channel interaction promoting.
Experimental results illustrate ASLE-SSM's superiority over existing state-of-the-art methods, with an inference speed 2.4 times faster than Transformer-based MST and saving 0.12 (M) of parameters.
arXiv Detail & Related papers (2024-08-01T15:14:10Z) - HTD-Mamba: Efficient Hyperspectral Target Detection with Pyramid State Space Model [5.505983410956103]
Hyperspectral target detection (HTD) identifies objects of interest from complex backgrounds at the pixel level.
This paper proposes an efficient self-supervised HTD method with a pyramid state space model (SSM), named HTD-Mamba.
Experiments conducted on four public datasets demonstrate that the proposed method outperforms state-of-the-art methods in both quantitative and qualitative evaluations.
arXiv Detail & Related papers (2024-07-09T13:21:26Z) - SSUMamba: Spatial-Spectral Selective State Space Model for Hyperspectral Image Denoising [13.1240990099267]
We introduce a memory-efficient spatial-spectralamba (SSUMamba) for HSI denoising.
Mamba is known for its remarkable long-range dependency modeling capabilities.
SSUMamba achieves superior denoising results with lower memory consumption per batch compared to transformer-based methods.
arXiv Detail & Related papers (2024-05-02T20:44:26Z) - Spectral-Spatial Mamba for Hyperspectral Image Classification [23.215920983979426]
spectral-spatial Mamba (SS-Mamba) is applied to hyperspectral image (HSI) classification.
The proposed SS-Mamba mainly consists of spectral-spatial token generation module and several stacked spectral-spatial Mamba blocks.
The experimental results conducted on widely used HSI datasets reveal that the proposed model achieves competitive results.
arXiv Detail & Related papers (2024-04-29T03:36:05Z) - Cross-Scope Spatial-Spectral Information Aggregation for Hyperspectral
Image Super-Resolution [47.12985199570964]
We propose a novel cross-scope spatial-spectral Transformer (CST) to investigate long-range spatial and spectral similarities for single hyperspectral image super-resolution.
Specifically, we devise cross-attention mechanisms in spatial and spectral dimensions to comprehensively model the long-range spatial-spectral characteristics.
Experiments over three hyperspectral datasets demonstrate that the proposed CST is superior to other state-of-the-art methods both quantitatively and visually.
arXiv Detail & Related papers (2023-11-29T03:38:56Z) - Object Detection in Hyperspectral Image via Unified Spectral-Spatial
Feature Aggregation [55.9217962930169]
We present S2ADet, an object detector that harnesses the rich spectral and spatial complementary information inherent in hyperspectral images.
S2ADet surpasses existing state-of-the-art methods, achieving robust and reliable results.
arXiv Detail & Related papers (2023-06-14T09:01:50Z) - Spectral Enhanced Rectangle Transformer for Hyperspectral Image
Denoising [64.11157141177208]
We propose a spectral enhanced rectangle Transformer to model the spatial and spectral correlation in hyperspectral images.
For the former, we exploit the rectangle self-attention horizontally and vertically to capture the non-local similarity in the spatial domain.
For the latter, we design a spectral enhancement module that is capable of extracting global underlying low-rank property of spatial-spectral cubes to suppress noise.
arXiv Detail & Related papers (2023-04-03T09:42:13Z) - Adaptive Linear Span Network for Object Skeleton Detection [56.78705071830965]
We propose adaptive linear span network (AdaLSN) to automatically configure and integrate scale-aware features for object skeleton detection.
AdaLSN substantiates its versatility by achieving significantly higher accuracy and latency trade-off.
It also demonstrates general applicability to image-to-mask tasks such as edge detection and road extraction.
arXiv Detail & Related papers (2020-11-08T12:51:14Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.