Cross-Attention in Coupled Unmixing Nets for Unsupervised Hyperspectral Super-Resolution

• URL: http://arxiv.org/abs/2007.05230v3
• Date: Sat, 1 Aug 2020 12:48:53 GMT
• Title: Cross-Attention in Coupled Unmixing Nets for Unsupervised Hyperspectral Super-Resolution
• Authors: Jing Yao, Danfeng Hong, Jocelyn Chanussot, Deyu Meng, Xiaoxiang Zhu, Zongben Xu
• Abstract summary: We propose a novel coupled unmixing network with a cross-attention mechanism, CUCaNet, to enhance the spatial resolution of HSI. Experiments are conducted on three widely-used HS-MS datasets in comparison with state-of-the-art HSI-SR models.
• Score: 79.97180849505294
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The recent advancement of deep learning techniques has made great progress on hyperspectral image super-resolution (HSI-SR). Yet the development of unsupervised deep networks remains challenging for this task. To this end, we propose a novel coupled unmixing network with a cross-attention mechanism, CUCaNet for short, to enhance the spatial resolution of HSI by means of higher-spatial-resolution multispectral image (MSI). Inspired by coupled spectral unmixing, a two-stream convolutional autoencoder framework is taken as backbone to jointly decompose MS and HS data into a spectrally meaningful basis and corresponding coefficients. CUCaNet is capable of adaptively learning spectral and spatial response functions from HS-MS correspondences by enforcing reasonable consistency assumptions on the networks. Moreover, a cross-attention module is devised to yield more effective spatial-spectral information transfer in networks. Extensive experiments are conducted on three widely-used HS-MS datasets in comparison with state-of-the-art HSI-SR models, demonstrating the superiority of the CUCaNet in the HSI-SR application. Furthermore, the codes and datasets will be available at: https://github.com/danfenghong/ECCV2020_CUCaNet.

Related papers

• HSR-KAN: Efficient Hyperspectral Image Super-Resolution via Kolmogorov-Arnold Networks [0.16385815610837165]
  We propose an efficient HSI super-resolution (HSI-SR) model to fuse a low-resolution HSI and a high-resolution multispectral image (HR-MSI) To achieve the effective integration of spatial information from HR-MSI, we design a fusion module based on KANs. As a channel attention module integrated with KANs, KAN-CAB enables networks to accurately simulate details of spectral sequences and spatial textures.
  arXiv  Detail & Related papers  (2024-08-24T02:51:51Z)
• Hybrid Convolutional and Attention Network for Hyperspectral Image Denoising [54.110544509099526]
  Hyperspectral image (HSI) denoising is critical for the effective analysis and interpretation of hyperspectral data. We propose a hybrid convolution and attention network (HCANet) to enhance HSI denoising. Experimental results on mainstream HSI datasets demonstrate the rationality and effectiveness of the proposed HCANet.
  arXiv  Detail & Related papers  (2024-03-15T07:18:43Z)
• ADASR: An Adversarial Auto-Augmentation Framework for Hyperspectral and Multispectral Data Fusion [54.668445421149364]
  Deep learning-based hyperspectral image (HSI) super-resolution aims to generate high spatial resolution HSI (HR-HSI) by fusing hyperspectral image (HSI) and multispectral image (MSI) with deep neural networks (DNNs) In this letter, we propose a novel adversarial automatic data augmentation framework ADASR that automatically optimize and augments HSI-MSI sample pairs to enrich data diversity for HSI-MSI fusion.
  arXiv  Detail & Related papers  (2023-10-11T07:30:37Z)
• Hyperspectral Image Super-Resolution via Dual-domain Network Based on Hybrid Convolution [6.3814314790000415]
  This paper proposes a novel HSI super-resolution algorithm, termed dual-domain network based on hybrid convolution (SRDNet) To capture inter-spectral self-similarity, a self-attention learning mechanism (HSL) is devised in the spatial domain. To further improve the perceptual quality of HSI, a frequency loss(HFL) is introduced to optimize the model in the frequency domain.
  arXiv  Detail & Related papers  (2023-04-10T13:51:28Z)
• RDRN: Recursively Defined Residual Network for Image Super-Resolution [58.64907136562178]
  Deep convolutional neural networks (CNNs) have obtained remarkable performance in single image super-resolution. We propose a novel network architecture which utilizes attention blocks efficiently.
  arXiv  Detail & Related papers  (2022-11-17T11:06:29Z)
• Deep Posterior Distribution-based Embedding for Hyperspectral Image Super-resolution [75.24345439401166]
  This paper focuses on how to embed the high-dimensional spatial-spectral information of hyperspectral (HS) images efficiently and effectively. We formulate HS embedding as an approximation of the posterior distribution of a set of carefully-defined HS embedding events. Then, we incorporate the proposed feature embedding scheme into a source-consistent super-resolution framework that is physically-interpretable. Experiments over three common benchmark datasets demonstrate that PDE-Net achieves superior performance over state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-05-30T06:59:01Z)
• Hyperspectral Image Super-Resolution with Spectral Mixup and Heterogeneous Datasets [99.92564298432387]
  This work studies Hyperspectral image (HSI) super-resolution (SR) HSI SR is characterized by high-dimensional data and a limited amount of training examples. This exacerbates the undesirable behaviors of neural networks such as memorization and sensitivity to out-of-distribution samples.
  arXiv  Detail & Related papers  (2021-01-19T12:19:53Z)
• Coupled Convolutional Neural Network with Adaptive Response Function Learning for Unsupervised Hyperspectral Super-Resolution [28.798775822331045]
  Hyperspectral super-resolution refers to fusing HSI and MSI to generate an image with both high spatial and high spectral resolutions. In this work, an unsupervised deep learning-based fusion method - HyCoNet - that can solve the problems in HSI-MSI fusion without the prior PSF and SRF information is proposed.
  arXiv  Detail & Related papers  (2020-07-28T06:17:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.