S$^2$Mamba: A Spatial-spectral State Space Model for Hyperspectral Image Classification
- URL: http://arxiv.org/abs/2404.18213v1
- Date: Sun, 28 Apr 2024 15:12:56 GMT
- Title: S$^2$Mamba: A Spatial-spectral State Space Model for Hyperspectral Image Classification
- Authors: Guanchun Wang, Xiangrong Zhang, Zelin Peng, Tianyang Zhang, Xiuping Jia, Licheng Jiao,
- Abstract summary: Land cover analysis using hyperspectral images (HSI) remains an open problem due to their low spatial resolution and complex spectral information.
We propose S$2$Mamba, a spatial-spectral state space model for hyperspectral image classification, to excavate spatial-spectral contextual features.
- Score: 47.32349241618756
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Land cover analysis using hyperspectral images (HSI) remains an open problem due to their low spatial resolution and complex spectral information. Recent studies are primarily dedicated to designing Transformer-based architectures for spatial-spectral long-range dependencies modeling, which is computationally expensive with quadratic complexity. Selective structured state space model (Mamba), which is efficient for modeling long-range dependencies with linear complexity, has recently shown promising progress. However, its potential in hyperspectral image processing that requires handling numerous spectral bands has not yet been explored. In this paper, we innovatively propose S$^2$Mamba, a spatial-spectral state space model for hyperspectral image classification, to excavate spatial-spectral contextual features, resulting in more efficient and accurate land cover analysis. In S$^2$Mamba, two selective structured state space models through different dimensions are designed for feature extraction, one for spatial, and the other for spectral, along with a spatial-spectral mixture gate for optimal fusion. More specifically, S$^2$Mamba first captures spatial contextual relations by interacting each pixel with its adjacent through a Patch Cross Scanning module and then explores semantic information from continuous spectral bands through a Bi-directional Spectral Scanning module. Considering the distinct expertise of the two attributes in homogenous and complicated texture scenes, we realize the Spatial-spectral Mixture Gate by a group of learnable matrices, allowing for the adaptive incorporation of representations learned across different dimensions. Extensive experiments conducted on HSI classification benchmarks demonstrate the superiority and prospect of S$^2$Mamba. The code will be available at: https://github.com/PURE-melo/S2Mamba.
Related papers
- HTD-Mamba: Efficient Hyperspectral Target Detection with Pyramid State Space Model [5.505983410956103]
Hyperspectral target detection (HTD) identifies objects of interest from complex backgrounds at the pixel level.
This paper proposes an efficient self-supervised HTD method with a pyramid state space model (SSM), named HTD-Mamba.
Experiments conducted on four public datasets demonstrate that the proposed method outperforms state-of-the-art methods in both quantitative and qualitative evaluations.
arXiv Detail & Related papers (2024-07-09T13:21:26Z) - SHMamba: Structured Hyperbolic State Space Model for Audio-Visual Question Answering [5.016335384639901]
Multi-modal input of Audio-Visual Question Answering (AVQA) makes feature extraction and fusion processes more challenging.
We propose SHMamba: Structured Hyperbolic State Space Model to integrate the advantages of hyperbolic geometry and state space models.
Our method demonstrates superiority among all current major methods and is more suitable for practical application scenarios.
arXiv Detail & Related papers (2024-06-14T08:43:31Z) - SSUMamba: Spatial-Spectral Selective State Space Model for Hyperspectral Image Denoising [13.1240990099267]
We introduce a memory-efficient spatial-spectralamba (SSUMamba) for HSI denoising.
Mamba is known for its remarkable long-range dependency modeling capabilities.
SSUMamba achieves superior denoising results with lower memory consumption per batch compared to transformer-based methods.
arXiv Detail & Related papers (2024-05-02T20:44:26Z) - Spectral-Spatial Mamba for Hyperspectral Image Classification [23.215920983979426]
spectral-spatial Mamba (SS-Mamba) is applied to hyperspectral image (HSI) classification.
The proposed SS-Mamba mainly consists of spectral-spatial token generation module and several stacked spectral-spatial Mamba blocks.
The experimental results conducted on widely used HSI datasets reveal that the proposed model achieves competitive results.
arXiv Detail & Related papers (2024-04-29T03:36:05Z) - Cross-Scope Spatial-Spectral Information Aggregation for Hyperspectral
Image Super-Resolution [47.12985199570964]
We propose a novel cross-scope spatial-spectral Transformer (CST) to investigate long-range spatial and spectral similarities for single hyperspectral image super-resolution.
Specifically, we devise cross-attention mechanisms in spatial and spectral dimensions to comprehensively model the long-range spatial-spectral characteristics.
Experiments over three hyperspectral datasets demonstrate that the proposed CST is superior to other state-of-the-art methods both quantitatively and visually.
arXiv Detail & Related papers (2023-11-29T03:38:56Z) - Object Detection in Hyperspectral Image via Unified Spectral-Spatial
Feature Aggregation [55.9217962930169]
We present S2ADet, an object detector that harnesses the rich spectral and spatial complementary information inherent in hyperspectral images.
S2ADet surpasses existing state-of-the-art methods, achieving robust and reliable results.
arXiv Detail & Related papers (2023-06-14T09:01:50Z) - Spectral Enhanced Rectangle Transformer for Hyperspectral Image
Denoising [64.11157141177208]
We propose a spectral enhanced rectangle Transformer to model the spatial and spectral correlation in hyperspectral images.
For the former, we exploit the rectangle self-attention horizontally and vertically to capture the non-local similarity in the spatial domain.
For the latter, we design a spectral enhancement module that is capable of extracting global underlying low-rank property of spatial-spectral cubes to suppress noise.
arXiv Detail & Related papers (2023-04-03T09:42:13Z) - Benchmarking Spatial Relationships in Text-to-Image Generation [102.62422723894232]
We investigate the ability of text-to-image models to generate correct spatial relationships among objects.
We present VISOR, an evaluation metric that captures how accurately the spatial relationship described in text is generated in the image.
Our experiments reveal a surprising finding that, although state-of-the-art T2I models exhibit high image quality, they are severely limited in their ability to generate multiple objects or the specified spatial relations between them.
arXiv Detail & Related papers (2022-12-20T06:03:51Z) - Adaptive Linear Span Network for Object Skeleton Detection [56.78705071830965]
We propose adaptive linear span network (AdaLSN) to automatically configure and integrate scale-aware features for object skeleton detection.
AdaLSN substantiates its versatility by achieving significantly higher accuracy and latency trade-off.
It also demonstrates general applicability to image-to-mask tasks such as edge detection and road extraction.
arXiv Detail & Related papers (2020-11-08T12:51:14Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.