TerraMAE: Learning Spatial-Spectral Representations from Hyperspectral Earth Observation Data via Adaptive Masked Autoencoders

• URL: http://arxiv.org/abs/2508.07020v1
• Date: Sat, 09 Aug 2025 15:32:22 GMT
• Title: TerraMAE: Learning Spatial-Spectral Representations from Hyperspectral Earth Observation Data via Adaptive Masked Autoencoders
• Authors: Tanjim Bin Faruk, Abdul Matin, Shrideep Pallickara, Sangmi Lee Pallickara,
• Abstract summary: We introduce TerraMAE, a novel HSI encoding framework designed to learn highly representative spatial-spectral embeddings for diverse geospatial analyses.<n>We demonstrate its effectiveness through superior spatial-spectral information preservation in high-fidelity image reconstruction.
• Score: 1.1999555634662633
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Hyperspectral satellite imagery offers sub-30 m views of Earth in hundreds of contiguous spectral bands, enabling fine-grained mapping of soils, crops, and land cover. While self-supervised Masked Autoencoders excel on RGB and low-band multispectral data, they struggle to exploit the intricate spatial-spectral correlations in 200+ band hyperspectral images. We introduce TerraMAE, a novel HSI encoding framework specifically designed to learn highly representative spatial-spectral embeddings for diverse geospatial analyses. TerraMAE features an adaptive channel grouping strategy, based on statistical reflectance properties to capture spectral similarities, and an enhanced reconstruction loss function that incorporates spatial and spectral quality metrics. We demonstrate TerraMAE's effectiveness through superior spatial-spectral information preservation in high-fidelity image reconstruction. Furthermore, we validate its practical utility and the quality of its learned representations through strong performance on three key downstream geospatial tasks: crop identification, land cover classification, and soil texture prediction.

Related papers

• Hyperspectral Adapter for Semantic Segmentation with Vision Foundation Models [18.24287471339871]
  Hyperspectral imaging (HSI) captures spatial information along with dense spectral measurements across numerous narrow wavelength bands.<n>Our architecture incorporates a spectral transformer and a spectrum-aware spatial prior module to extract rich spatial-spectral features.<n>Our architecture achieves state-of-the-art semantic segmentation performance while directly using HSI inputs, outperforming both vision-based and hyperspectral segmentation methods.
  arXiv  Detail & Related papers  (2025-09-24T13:32:07Z)
• TerraFM: A Scalable Foundation Model for Unified Multisensor Earth Observation [65.74990259650984]
  We introduce TerraFM, a scalable self-supervised learning model that leverages globally distributed Sentinel-1 and Sentinel-2 imagery.<n>Our training strategy integrates local-global contrastive learning and introduces a dual-centering mechanism.<n>TerraFM achieves strong generalization on both classification and segmentation tasks, outperforming prior models on GEO-Bench and Copernicus-Bench.
  arXiv  Detail & Related papers  (2025-06-06T17:59:50Z)
• Towards Scalable Foundation Model for Multi-modal and Hyperspectral Geospatial Data [14.104497777255137]
  We introduce Low-rank Efficient Spatial-Spectral Vision Transformer with three key innovations.<n>We pretrain LESS ViT using a Hyperspectral Masked Autoencoder framework with integrated positional and channel masking strategies.<n> Experimental results demonstrate that our proposed method achieves competitive performance against state-of-the-art multi-modal geospatial foundation models.
  arXiv  Detail & Related papers  (2025-03-17T05:42:19Z)
• SpectralEarth: Training Hyperspectral Foundation Models at Scale [47.93167977587301]
  We introduce SpectralEarth, a large-scale multitemporal dataset designed to pretrain hyperspectral foundation models.<n>We pretrain a series of foundation models on SpectralEarth, integrating a spectral adapter into classical vision backbones.<n>In tandem, we construct nine downstream datasets for land-cover, crop-type mapping, and tree-species classification.
  arXiv  Detail & Related papers  (2024-08-15T22:55:59Z)
• S2RC-GCN: A Spatial-Spectral Reliable Contrastive Graph Convolutional Network for Complex Land Cover Classification Using Hyperspectral Images [10.579474650543471]
  This study proposes a novel spatial-spectral reliable contrastive graph convolutional classification framework named S2RC-GCN. Specifically, we fused the spectral and spatial features extracted by the 1D- and 2D-encoder, and the 2D-encoder includes an attention model to automatically extract important information. We then leveraged the fused high-level features to construct graphs and fed the resulting graphs into the GCNs to determine more effective graph representations.
  arXiv  Detail & Related papers  (2024-04-01T07:17:02Z)
• Toulouse Hyperspectral Data Set: a benchmark data set to assess semi-supervised spectral representation learning and pixel-wise classification techniques [0.6384650391969042]
  We release in this paper the Toulouse Hyperspectral Data Set to meet key issues in spectral representation learning and classification over large-scale hyperspectral images with very few labeled pixels. We discuss and experiment self-supervised techniques for spectral representation learning, including the Masked Autoencoder, and establish a baseline for pixel-wise classification achieving 85% overall accuracy and 77% F1 score.
  arXiv  Detail & Related papers  (2023-11-15T10:49:15Z)
• Object Detection in Hyperspectral Image via Unified Spectral-Spatial Feature Aggregation [55.9217962930169]
  We present S2ADet, an object detector that harnesses the rich spectral and spatial complementary information inherent in hyperspectral images. S2ADet surpasses existing state-of-the-art methods, achieving robust and reliable results.
  arXiv  Detail & Related papers  (2023-06-14T09:01:50Z)
• DCN-T: Dual Context Network with Transformer for Hyperspectral Image Classification [109.09061514799413]
  Hyperspectral image (HSI) classification is challenging due to spatial variability caused by complex imaging conditions. We propose a tri-spectral image generation pipeline that transforms HSI into high-quality tri-spectral images. Our proposed method outperforms state-of-the-art methods for HSI classification.
  arXiv  Detail & Related papers  (2023-04-19T18:32:52Z)
• SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagery [74.82821342249039]
  We present SatMAE, a pre-training framework for temporal or multi-spectral satellite imagery based on Masked Autoencoder (MAE) To leverage temporal information, we include a temporal embedding along with independently masking image patches across time.
  arXiv  Detail & Related papers  (2022-07-17T01:35:29Z)
• Mask-guided Spectral-wise Transformer for Efficient Hyperspectral Image Reconstruction [127.20208645280438]
  Hyperspectral image (HSI) reconstruction aims to recover the 3D spatial-spectral signal from a 2D measurement. Modeling the inter-spectra interactions is beneficial for HSI reconstruction. Mask-guided Spectral-wise Transformer (MST) proposes a novel framework for HSI reconstruction.
  arXiv  Detail & Related papers  (2021-11-15T16:59:48Z)
• Spatial-Spectral Residual Network for Hyperspectral Image Super-Resolution [82.1739023587565]
  We propose a novel spectral-spatial residual network for hyperspectral image super-resolution (SSRNet) Our method can effectively explore spatial-spectral information by using 3D convolution instead of 2D convolution, which enables the network to better extract potential information. In each unit, we employ spatial and temporal separable 3D convolution to extract spatial and spectral information, which not only reduces unaffordable memory usage and high computational cost, but also makes the network easier to train.
  arXiv  Detail & Related papers  (2020-01-14T03:34:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.