A Latent Encoder Coupled Generative Adversarial Network (LE-GAN) for Efficient Hyperspectral Image Super-resolution

• URL: http://arxiv.org/abs/2111.08685v1
• Date: Tue, 16 Nov 2021 18:40:19 GMT
• Title: A Latent Encoder Coupled Generative Adversarial Network (LE-GAN) for Efficient Hyperspectral Image Super-resolution
• Authors: Yue Shi, Liangxiu Han, Lianghao Han, Sheng Chang, Tongle Hu, Darren Dancey
• Abstract summary: generative adversarial network (GAN) has proven to be an effective deep learning framework for image super-resolution. To alleviate the problem of mode collapse, this work has proposed a novel GAN model coupled with a latent encoder (LE-GAN) LE-GAN can map the generated spectral-spatial features from the image space to the latent space and produce a coupling component to regularise the generated samples.
• Score: 3.1023808510465627
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Realistic hyperspectral image (HSI) super-resolution (SR) techniques aim to generate a high-resolution (HR) HSI with higher spectral and spatial fidelity from its low-resolution (LR) counterpart. The generative adversarial network (GAN) has proven to be an effective deep learning framework for image super-resolution. However, the optimisation process of existing GAN-based models frequently suffers from the problem of mode collapse, leading to the limited capacity of spectral-spatial invariant reconstruction. This may cause the spectral-spatial distortion on the generated HSI, especially with a large upscaling factor. To alleviate the problem of mode collapse, this work has proposed a novel GAN model coupled with a latent encoder (LE-GAN), which can map the generated spectral-spatial features from the image space to the latent space and produce a coupling component to regularise the generated samples. Essentially, we treat an HSI as a high-dimensional manifold embedded in a latent space. Thus, the optimisation of GAN models is converted to the problem of learning the distributions of high-resolution HSI samples in the latent space, making the distributions of the generated super-resolution HSIs closer to those of their original high-resolution counterparts. We have conducted experimental evaluations on the model performance of super-resolution and its capability in alleviating mode collapse. The proposed approach has been tested and validated based on two real HSI datasets with different sensors (i.e. AVIRIS and UHD-185) for various upscaling factors and added noise levels, and compared with the state-of-the-art super-resolution models (i.e. HyCoNet, LTTR, BAGAN, SR- GAN, WGAN).

Related papers

• HSIGene: A Foundation Model For Hyperspectral Image Generation [46.745198868466545]
  Hyperspectral image (HSI) plays a vital role in various fields such as agriculture and environmental monitoring. Due to the expensive acquisition cost, the number of hyperspectral images is limited, degenerating the performance of downstream tasks. We propose HSIGene, a novel HSI generation foundation model which is based on latent diffusion and supports multi-condition control. Experiments demonstrate that the proposed model is capable of generating a vast quantity of realistic HSIs for downstream tasks such as denoising and super-resolution.
  arXiv  Detail & Related papers  (2024-09-19T05:17:44Z)
• Hybrid Convolutional and Attention Network for Hyperspectral Image Denoising [54.110544509099526]
  Hyperspectral image (HSI) denoising is critical for the effective analysis and interpretation of hyperspectral data. We propose a hybrid convolution and attention network (HCANet) to enhance HSI denoising. Experimental results on mainstream HSI datasets demonstrate the rationality and effectiveness of the proposed HCANet.
  arXiv  Detail & Related papers  (2024-03-15T07:18:43Z)
• Physics-Inspired Degradation Models for Hyperspectral Image Fusion [61.743696362028246]
  Most fusion methods solely focus on the fusion algorithm itself and overlook the degradation models. We propose physics-inspired degradation models (PIDM) to model the degradation of LR-HSI and HR-MSI. Our proposed PIDM can boost the fusion performance of existing fusion methods in practical scenarios.
  arXiv  Detail & Related papers  (2024-02-04T09:07:28Z)
• A Spectral Diffusion Prior for Hyperspectral Image Super-Resolution [14.405562058304074]
  Fusion-based hyperspectral image (HSI) super-resolution aims to produce a high-spatial-resolution HSI by fusing a low-spatial-resolution HSI and a high-spatial-resolution multispectral image. Motivated by the success of diffusion models, we propose a novel spectral diffusion prior for fusion-based HSI super-resolution.
  arXiv  Detail & Related papers  (2023-11-15T13:40:58Z)
• SSIF: Learning Continuous Image Representation for Spatial-Spectral Super-Resolution [73.46167948298041]
  We propose a neural implicit model that represents an image as a function of both continuous pixel coordinates in the spatial domain and continuous wavelengths in the spectral domain. We show that SSIF generalizes well to both unseen spatial resolutions and spectral resolutions. It can generate high-resolution images that improve the performance of downstream tasks by 1.7%-7%.
  arXiv  Detail & Related papers  (2023-09-30T15:23:30Z)
• ESSAformer: Efficient Transformer for Hyperspectral Image Super-resolution [76.7408734079706]
  Single hyperspectral image super-resolution (single-HSI-SR) aims to restore a high-resolution hyperspectral image from a low-resolution observation. We propose ESSAformer, an ESSA attention-embedded Transformer network for single-HSI-SR with an iterative refining structure.
  arXiv  Detail & Related papers  (2023-07-26T07:45:14Z)
• Hyperspectral Image Super-Resolution via Dual-domain Network Based on Hybrid Convolution [6.3814314790000415]
  This paper proposes a novel HSI super-resolution algorithm, termed dual-domain network based on hybrid convolution (SRDNet) To capture inter-spectral self-similarity, a self-attention learning mechanism (HSL) is devised in the spatial domain. To further improve the perceptual quality of HSI, a frequency loss(HFL) is introduced to optimize the model in the frequency domain.
  arXiv  Detail & Related papers  (2023-04-10T13:51:28Z)
• HDNet: High-resolution Dual-domain Learning for Spectral Compressive Imaging [138.04956118993934]
  We propose a high-resolution dual-domain learning network (HDNet) for HSI reconstruction. On the one hand, the proposed HR spatial-spectral attention module with its efficient feature fusion provides continuous and fine pixel-level features. On the other hand, frequency domain learning (FDL) is introduced for HSI reconstruction to narrow the frequency domain discrepancy.
  arXiv  Detail & Related papers  (2022-03-04T06:37:45Z)
• Hyperspectral Image Super-resolution with Deep Priors and Degradation Model Inversion [6.0622962428871885]
  Fusion-based hyperspectral image (HSI) super-resolution is attracting increasing attention. Deep learning architectures have been used to address the HSI super-resolution problem. We propose a method that makes use of the linear degradation model in the data-fidelity term of the objective function.
  arXiv  Detail & Related papers  (2022-01-24T18:17:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.