SSFT: A Lightweight Spectral-Spatial Fusion Transformer for Generic Hyperspectral Classification

• URL: http://arxiv.org/abs/2604.15828v1
• Date: Fri, 17 Apr 2026 08:31:25 GMT
• Title: SSFT: A Lightweight Spectral-Spatial Fusion Transformer for Generic Hyperspectral Classification
• Abstract summary: We propose the lightweight Spectral-Spatial Fusion Transformer (SSFT), which factorizes representation learning into spectral and spatial pathways.<n>We evaluate our SSFT on the challenging HSI-Benchmark, a heterogeneous multi-dataset benchmark covering earth observation, fruit condition assessment, and fine-grained material recognition.
• Score: 42.09644109233563
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Hyperspectral imaging enables fine-grained recognition of materials by capturing rich spectral signatures, but learning robust classifiers is challenging due to high dimensionality, spectral redundancy, limited labeled data, and strong domain shifts. Beyond earth observation, labeled HSI data is often scarce and imbalanced, motivating compact models for generic hyperspectral classification across diverse acquisition regimes. We propose the lightweight Spectral-Spatial Fusion Transformer (SSFT), which factorizes representation learning into spectral and spatial pathways and integrates them via cross-attention to capture complementary wavelength-dependent and structural information. We evaluate our SSFT on the challenging HSI-Benchmark, a heterogeneous multi-dataset benchmark covering earth observation, fruit condition assessment, and fine-grained material recognition. SSFT achieves state-of-the-art overall performance, ranking first while using less than 2% of the parameters of the previous leading method. We further evaluate transfer to the substantially larger SpectralEarth benchmark under the official protocol, where SSFT remains competitive despite its compact size. Ablation studies show that both spectral and spatial pathways are crucial, with spatial modeling contributing most, and that SSFT remains robust without data augmentation.

Related papers

• LESSViT: Robust Hyperspectral Representation Learning under Spectral Configuration Shift [29.791943499456426]
  Low-rank Efficient Spatial-Spectral ViT (LESSViT) is a sensor-flexible architecture for cross-spectral generalization.<n>We introduce a hyperspectral masked autoencoder (HyperMAE) with decoupled spatial-spectral masking and hierarchical channel sampling.
  arXiv  Detail & Related papers  (2026-05-18T15:22:26Z)
• Representative Spectral Correlation Network for Multi-source Remote Sensing Image Classification [44.20355634294824]
  Representative Spectral Correlation Network (RSCNet) is a novel multi-source image classification framework.<n>RSCNet addresses challenges through spectral selection and adaptive interaction.<n> Experiments on three public benchmark datasets demonstrate that RSCNet achieves superior performance compared with state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-04-30T02:10:38Z)
• HSSDCT: Factorized Spatial-Spectral Correlation for Hyperspectral Image Fusion [11.994592153994482]
  Hyperspectral image (HSI) fusion aims to reconstruct a high-resolution HSI (HR-HSI) by combining the rich spectral information of a low-resolution HSI with the fine details of a high-resolution multispectral image (HR-MSI)<n>Recent deep learning methods have achieved notable progress, but they still suffer from limited receptive fields, redundant spectral bands, and the quadratic complexity of self-attention.<n>We propose the Hierarchical Spatial-Spectral Dense Correlation Network (HSSDCT) to overcome these challenges.
  arXiv  Detail & Related papers  (2026-01-31T03:24:03Z)
• Spectral Discrepancy and Cross-modal Semantic Consistency Learning for Object Detection in Hyperspectral Image [40.38555448650773]
  Hyperspectral images with high spectral resolution provide new insights into recognizing subtle differences in similar substances.<n> object detection in hyperspectral images faces significant challenges in intra- and inter-class similarity due to the spatial differences in hyperspectral inter-bands.<n>We propose a novel network termed textbfSpectral textbfDiscrepancy and textbfCross-textbfModal semantic consistency learning (SDCM)<n>Our proposed method achieves state-of-the-art performance when compared with other ones.
  arXiv  Detail & Related papers  (2025-12-20T07:03:09Z)
• Unleashing Correlation and Continuity for Hyperspectral Reconstruction from RGB Images [64.80875911446937]
  We propose a Correlation and Continuity Network (CCNet) for HSI reconstruction from RGB images.<n>For the correlation of local spectrum, we introduce the Group-wise Spectral Correlation Modeling (GrSCM) module.<n>For the continuity of global spectrum, we design the Neighborhood-wise Spectral Continuity Modeling (NeSCM) module.
  arXiv  Detail & Related papers  (2025-01-02T15:14:40Z)
• HyperSIGMA: Hyperspectral Intelligence Comprehension Foundation Model [88.13261547704444]
  Hyper SIGMA is a vision transformer-based foundation model that unifies HSI interpretation across tasks and scenes.<n>In addition, we construct a large-scale hyperspectral dataset, HyperGlobal-450K, for pre-training, which contains about 450K hyperspectral images.
  arXiv  Detail & Related papers  (2024-06-17T13:22:58Z)
• SSF-Net: Spatial-Spectral Fusion Network with Spectral Angle Awareness for Hyperspectral Object Tracking [21.664141982246598]
  Hyperspectral video (HSV) offers valuable spatial, spectral, and temporal information simultaneously.<n>Existing methods primarily focus on band regrouping and rely on RGB trackers for feature extraction.<n>In this paper, a spatial-spectral fusion network with spectral angle awareness (SST-Net) is proposed for hyperspectral (HS) object tracking.
  arXiv  Detail & Related papers  (2024-03-09T09:37:13Z)
• ESSAformer: Efficient Transformer for Hyperspectral Image Super-resolution [76.7408734079706]
  Single hyperspectral image super-resolution (single-HSI-SR) aims to restore a high-resolution hyperspectral image from a low-resolution observation. We propose ESSAformer, an ESSA attention-embedded Transformer network for single-HSI-SR with an iterative refining structure.
  arXiv  Detail & Related papers  (2023-07-26T07:45:14Z)
• Unsupervised Machine Learning for Exploratory Data Analysis of Exoplanet Transmission Spectra [68.8204255655161]
  We focus on unsupervised techniques for analyzing spectral data from transiting exoplanets. We show that there is a high degree of correlation in the spectral data, which calls for appropriate low-dimensional representations. We uncover interesting structures in the principal component basis, namely, well-defined branches corresponding to different chemical regimes.
  arXiv  Detail & Related papers  (2022-01-07T22:26:33Z)
• Mask-guided Spectral-wise Transformer for Efficient Hyperspectral Image Reconstruction [127.20208645280438]
  Hyperspectral image (HSI) reconstruction aims to recover the 3D spatial-spectral signal from a 2D measurement. Modeling the inter-spectra interactions is beneficial for HSI reconstruction. Mask-guided Spectral-wise Transformer (MST) proposes a novel framework for HSI reconstruction.
  arXiv  Detail & Related papers  (2021-11-15T16:59:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.