SaaFormer: Spectral-spatial Axial Aggregation Transformer for
Hyperspectral Image Classification
- URL: http://arxiv.org/abs/2306.16759v2
- Date: Tue, 4 Jul 2023 05:51:58 GMT
- Title: SaaFormer: Spectral-spatial Axial Aggregation Transformer for
Hyperspectral Image Classification
- Authors: Enzhe Zhao, Zhichang Guo, Yao Li, Dazhi Zhang
- Abstract summary: Hyperspectral images (HSI) captured from earth observing satellites and aircraft is becoming increasingly important for applications in agriculture, environmental monitoring, mining, etc.
Due to the limited available hyperspectral datasets, the pixel-wise random sampling is the most commonly used training-test dataset partition approach.
We propose a block-wise sampling method to minimize the potential for data leakage.
- Score: 2.4723464787484812
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Hyperspectral images (HSI) captured from earth observing satellites and
aircraft is becoming increasingly important for applications in agriculture,
environmental monitoring, mining, etc. Due to the limited available
hyperspectral datasets, the pixel-wise random sampling is the most commonly
used training-test dataset partition approach, which has significant overlap
between samples in training and test datasets. Furthermore, our experimental
observations indicates that regions with larger overlap often exhibit higher
classification accuracy. Consequently, the pixel-wise random sampling approach
poses a risk of data leakage. Thus, we propose a block-wise sampling method to
minimize the potential for data leakage. Our experimental findings also confirm
the presence of data leakage in models such as 2DCNN. Further, We propose a
spectral-spatial axial aggregation transformer model, namely SaaFormer, to
address the challenges associated with hyperspectral image classifier that
considers HSI as long sequential three-dimensional images. The model comprises
two primary components: axial aggregation attention and multi-level
spectral-spatial extraction. The axial aggregation attention mechanism
effectively exploits the continuity and correlation among spectral bands at
each pixel position in hyperspectral images, while aggregating spatial
dimension features. This enables SaaFormer to maintain high precision even
under block-wise sampling. The multi-level spectral-spatial extraction
structure is designed to capture the sensitivity of different material
components to specific spectral bands, allowing the model to focus on a broader
range of spectral details. The results on six publicly available datasets
demonstrate that our model exhibits comparable performance when using random
sampling, while significantly outperforming other methods when employing
block-wise sampling partition.
Related papers
- Datacube segmentation via Deep Spectral Clustering [76.48544221010424]
Extended Vision techniques often pose a challenge in their interpretation.
The huge dimensionality of data cube spectra poses a complex task in its statistical interpretation.
In this paper, we explore the possibility of applying unsupervised clustering methods in encoded space.
A statistical dimensional reduction is performed by an ad hoc trained (Variational) AutoEncoder, while the clustering process is performed by a (learnable) iterative K-Means clustering algorithm.
arXiv Detail & Related papers (2024-01-31T09:31:28Z) - ESSAformer: Efficient Transformer for Hyperspectral Image
Super-resolution [76.7408734079706]
Single hyperspectral image super-resolution (single-HSI-SR) aims to restore a high-resolution hyperspectral image from a low-resolution observation.
We propose ESSAformer, an ESSA attention-embedded Transformer network for single-HSI-SR with an iterative refining structure.
arXiv Detail & Related papers (2023-07-26T07:45:14Z) - Object Detection in Hyperspectral Image via Unified Spectral-Spatial
Feature Aggregation [55.9217962930169]
We present S2ADet, an object detector that harnesses the rich spectral and spatial complementary information inherent in hyperspectral images.
S2ADet surpasses existing state-of-the-art methods, achieving robust and reliable results.
arXiv Detail & Related papers (2023-06-14T09:01:50Z) - HyperPCA: a Powerful Tool to Extract Elemental Maps from Noisy Data
Obtained in LIBS Mapping of Materials [7.648784748888189]
We introduce HyperPCA, a new analysis tool for hyperspectral images based on a sparse representation of the data.
We show that the method presents advantages both in quantity and quality of the information recovered, thus improving the physico-chemical characterisation of analysed surfaces.
arXiv Detail & Related papers (2021-11-30T07:52:44Z) - Spectral Splitting and Aggregation Network for Hyperspectral Face
Super-Resolution [82.59267937569213]
High-resolution (HR) hyperspectral face image plays an important role in face related computer vision tasks under uncontrolled conditions.
In this paper, we investigate how to adapt the deep learning techniques to hyperspectral face image super-resolution.
We present a spectral splitting and aggregation network (SSANet) for HFSR with limited training samples.
arXiv Detail & Related papers (2021-08-31T02:13:00Z) - Hyperspectral Band Selection for Multispectral Image Classification with
Convolutional Networks [0.0]
We propose a novel band selection method to select a reduced set of wavelengths from hyperspectral images.
We show that our method produces more suitable results for a multispectral sensor design.
arXiv Detail & Related papers (2021-06-01T17:24:35Z) - Spatial-Phase Shallow Learning: Rethinking Face Forgery Detection in
Frequency Domain [88.7339322596758]
We present a novel Spatial-Phase Shallow Learning (SPSL) method, which combines spatial image and phase spectrum to capture the up-sampling artifacts of face forgery.
SPSL can achieve the state-of-the-art performance on cross-datasets evaluation as well as multi-class classification and obtain comparable results on single dataset evaluation.
arXiv Detail & Related papers (2021-03-02T16:45:08Z) - Fusion of Dual Spatial Information for Hyperspectral Image
Classification [26.304992631350114]
A novel hyperspectral image classification framework using the fusion of dual spatial information is proposed.
Experiments performed on three data sets from different scenes illustrate that the proposed method can outperform other state-of-the-art classification techniques.
arXiv Detail & Related papers (2020-10-23T12:20:18Z) - Learning Spatial-Spectral Prior for Super-Resolution of Hyperspectral
Imagery [79.69449412334188]
In this paper, we investigate how to adapt state-of-the-art residual learning based single gray/RGB image super-resolution approaches.
We introduce a spatial-spectral prior network (SSPN) to fully exploit the spatial information and the correlation between the spectra of the hyperspectral data.
Experimental results on some hyperspectral images demonstrate that the proposed SSPSR method enhances the details of the recovered high-resolution hyperspectral images.
arXiv Detail & Related papers (2020-05-18T14:25:50Z) - Hyperspectral Image Classification Based on Sparse Modeling of Spectral
Blocks [6.99674326582747]
A sparse modeling framework is proposed for hyperspectral image classification.
The proposed method leads to a robust sparse modeling of hyperspectral images and improves the classification accuracy.
arXiv Detail & Related papers (2020-05-17T08:18:13Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.