Hybrid Spectral Denoising Transformer with Guided Attention

• URL: http://arxiv.org/abs/2303.09040v2
• Date: Tue, 8 Aug 2023 16:14:32 GMT
• Title: Hybrid Spectral Denoising Transformer with Guided Attention
• Authors: Zeqiang Lai, Chenggang Yan, Ying Fu
• Abstract summary: We present a Hybrid Spectral Denoising Transformer (HSDT) for hyperspectral image denoising. Our HSDT significantly outperforms the existing state-of-the-art methods while maintaining low computational overhead.
• Score: 34.34075175179669
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this paper, we present a Hybrid Spectral Denoising Transformer (HSDT) for hyperspectral image denoising. Challenges in adapting transformer for HSI arise from the capabilities to tackle existing limitations of CNN-based methods in capturing the global and local spatial-spectral correlations while maintaining efficiency and flexibility. To address these issues, we introduce a hybrid approach that combines the advantages of both models with a Spatial-Spectral Separable Convolution (S3Conv), Guided Spectral Self-Attention (GSSA), and Self-Modulated Feed-Forward Network (SM-FFN). Our S3Conv works as a lightweight alternative to 3D convolution, which extracts more spatial-spectral correlated features while keeping the flexibility to tackle HSIs with an arbitrary number of bands. These features are then adaptively processed by GSSA which per-forms 3D self-attention across the spectral bands, guided by a set of learnable queries that encode the spectral signatures. This not only enriches our model with powerful capabilities for identifying global spectral correlations but also maintains linear complexity. Moreover, our SM-FFN proposes the self-modulation that intensifies the activations of more informative regions, which further strengthens the aggregated features. Extensive experiments are conducted on various datasets under both simulated and real-world noise, and it shows that our HSDT significantly outperforms the existing state-of-the-art methods while maintaining low computational overhead. Code is at https: //github.com/Zeqiang-Lai/HSDT.

Related papers

• Hybrid Convolutional and Attention Network for Hyperspectral Image Denoising [54.110544509099526]
  Hyperspectral image (HSI) denoising is critical for the effective analysis and interpretation of hyperspectral data. We propose a hybrid convolution and attention network (HCANet) to enhance HSI denoising. Experimental results on mainstream HSI datasets demonstrate the rationality and effectiveness of the proposed HCANet.
  arXiv  Detail & Related papers  (2024-03-15T07:18:43Z)
• Hyperspectral Image Denoising via Self-Modulating Convolutional Neural Networks [15.700048595212051]
  We introduce a self-modulating convolutional neural network which utilizes correlated spectral and spatial information. At the core of the model lies a novel block, which allows the network to transform the features in an adaptive manner based on the adjacent spectral data. Experimental analysis on both synthetic and real data shows that the proposed SM-CNN outperforms other state-of-the-art HSI denoising methods.
  arXiv  Detail & Related papers  (2023-09-15T06:57:43Z)
• ESSAformer: Efficient Transformer for Hyperspectral Image Super-resolution [76.7408734079706]
  Single hyperspectral image super-resolution (single-HSI-SR) aims to restore a high-resolution hyperspectral image from a low-resolution observation. We propose ESSAformer, an ESSA attention-embedded Transformer network for single-HSI-SR with an iterative refining structure.
  arXiv  Detail & Related papers  (2023-07-26T07:45:14Z)
• DCN-T: Dual Context Network with Transformer for Hyperspectral Image Classification [109.09061514799413]
  Hyperspectral image (HSI) classification is challenging due to spatial variability caused by complex imaging conditions. We propose a tri-spectral image generation pipeline that transforms HSI into high-quality tri-spectral images. Our proposed method outperforms state-of-the-art methods for HSI classification.
  arXiv  Detail & Related papers  (2023-04-19T18:32:52Z)
• Spectral Enhanced Rectangle Transformer for Hyperspectral Image Denoising [64.11157141177208]
  We propose a spectral enhanced rectangle Transformer to model the spatial and spectral correlation in hyperspectral images. For the former, we exploit the rectangle self-attention horizontally and vertically to capture the non-local similarity in the spatial domain. For the latter, we design a spectral enhancement module that is capable of extracting global underlying low-rank property of spatial-spectral cubes to suppress noise.
  arXiv  Detail & Related papers  (2023-04-03T09:42:13Z)
• Improved Quasi-Recurrent Neural Network for Hyperspectral Image Denoising [9.723155514555765]
  We show that with a few simple modifications, the performance of QRNN3D could be substantially improved further. We introduce an adaptive fusion module to replace its vanilla additive skip connection to better fuse the features of the encoder and decoder. Experimental results on various noise settings demonstrate the effectiveness and superior performance of our method.
  arXiv  Detail & Related papers  (2022-11-27T12:38:03Z)
• Mask-guided Spectral-wise Transformer for Efficient Hyperspectral Image Reconstruction [127.20208645280438]
  Hyperspectral image (HSI) reconstruction aims to recover the 3D spatial-spectral signal from a 2D measurement. Modeling the inter-spectra interactions is beneficial for HSI reconstruction. Mask-guided Spectral-wise Transformer (MST) proposes a novel framework for HSI reconstruction.
  arXiv  Detail & Related papers  (2021-11-15T16:59:48Z)
• 3D Quasi-Recurrent Neural Network for Hyperspectral Image Denoising [25.641742612227148]
  3D convolution is utilized to extract structural-spectral correlation in an HS image. alternating global directional structure is introduced to eliminate causal dependency. experiments on HSI denoising demonstrate significant improvement over state-of-the-arts computation.
  arXiv  Detail & Related papers  (2020-03-10T06:14:53Z)
• Spatial-Spectral Residual Network for Hyperspectral Image Super-Resolution [82.1739023587565]
  We propose a novel spectral-spatial residual network for hyperspectral image super-resolution (SSRNet) Our method can effectively explore spatial-spectral information by using 3D convolution instead of 2D convolution, which enables the network to better extract potential information. In each unit, we employ spatial and temporal separable 3D convolution to extract spatial and spectral information, which not only reduces unaffordable memory usage and high computational cost, but also makes the network easier to train.
  arXiv  Detail & Related papers  (2020-01-14T03:34:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.