Hyperspectral Pansharpening Based on Improved Deep Image Prior and Residual Reconstruction

• URL: http://arxiv.org/abs/2107.02630v1
• Date: Tue, 6 Jul 2021 14:11:03 GMT
• Title: Hyperspectral Pansharpening Based on Improved Deep Image Prior and Residual Reconstruction
• Authors: Wele Gedara Chaminda Bandara, Jeya Maria Jose Valanarasu, Vishal M. Patel
• Abstract summary: Hyperspectral pansharpening aims to synthesize a low-resolution hyperspectral image (LR-HSI) with a registered panchromatic image (PAN) to generate an enhanced HSI with high spectral and spatial resolution. Recently proposed HS pansharpening methods have obtained remarkable results using deep convolutional networks (ConvNets) We propose a novel over-complete network, called HyperKite, which focuses on learning high-level features by constraining the receptive from increasing in the deep layers.
• Score: 64.10636296274168
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Hyperspectral pansharpening aims to synthesize a low-resolution hyperspectral image (LR-HSI) with a registered panchromatic image (PAN) to generate an enhanced HSI with high spectral and spatial resolution. Recently proposed HS pansharpening methods have obtained remarkable results using deep convolutional networks (ConvNets), which typically consist of three steps: (1) up-sampling the LR-HSI, (2) predicting the residual image via a ConvNet, and (3) obtaining the final fused HSI by adding the outputs from first and second steps. Recent methods have leveraged Deep Image Prior (DIP) to up-sample the LR-HSI due to its excellent ability to preserve both spatial and spectral information, without learning from large data sets. However, we observed that the quality of up-sampled HSIs can be further improved by introducing an additional spatial-domain constraint to the conventional spectral-domain energy function. We define our spatial-domain constraint as the $L_1$ distance between the predicted PAN image and the actual PAN image. To estimate the PAN image of the up-sampled HSI, we also propose a learnable spectral response function (SRF). Moreover, we noticed that the residual image between the up-sampled HSI and the reference HSI mainly consists of edge information and very fine structures. In order to accurately estimate fine information, we propose a novel over-complete network, called HyperKite, which focuses on learning high-level features by constraining the receptive from increasing in the deep layers. We perform experiments on three HSI datasets to demonstrate the superiority of our DIP-HyperKite over the state-of-the-art pansharpening methods. The deployment codes, pre-trained models, and final fusion outputs of our DIP-HyperKite and the methods used for the comparisons will be publicly made available at https://github.com/wgcban/DIP-HyperKite.git.

Related papers

• Superpixel-based and Spatially-regularized Diffusion Learning for Unsupervised Hyperspectral Image Clustering [4.643572021927615]
  This paper introduces a novel unsupervised HSI clustering algorithm, Superpixel-based and Spatially-regularized Diffusion Learning (S2DL) S2DL incorporates rich spatial information encoded in HSIs into diffusion geometry-based clustering. S2DL's performance is illustrated with extensive experiments on three publicly available, real-world HSIs.
  arXiv  Detail & Related papers  (2023-12-24T09:54:40Z)
• PC-GANs: Progressive Compensation Generative Adversarial Networks for Pan-sharpening [50.943080184828524]
  We propose a novel two-step model for pan-sharpening that sharpens the MS image through the progressive compensation of the spatial and spectral information. The whole model is composed of triple GANs, and based on the specific architecture, a joint compensation loss function is designed to enable the triple GANs to be trained simultaneously.
  arXiv  Detail & Related papers  (2022-07-29T03:09:21Z)
• Deep Posterior Distribution-based Embedding for Hyperspectral Image Super-resolution [75.24345439401166]
  This paper focuses on how to embed the high-dimensional spatial-spectral information of hyperspectral (HS) images efficiently and effectively. We formulate HS embedding as an approximation of the posterior distribution of a set of carefully-defined HS embedding events. Then, we incorporate the proposed feature embedding scheme into a source-consistent super-resolution framework that is physically-interpretable. Experiments over three common benchmark datasets demonstrate that PDE-Net achieves superior performance over state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-05-30T06:59:01Z)
• HyperTransformer: A Textural and Spectral Feature Fusion Transformer for Pansharpening [60.89777029184023]
  Pansharpening aims to fuse a registered high-resolution panchromatic image (PAN) with a low-resolution hyperspectral image (LR-HSI) to generate an enhanced HSI with high spectral and spatial resolution. Existing pansharpening approaches neglect using an attention mechanism to transfer HR texture features from PAN to LR-HSI features, resulting in spatial and spectral distortions. We present a novel attention mechanism for pansharpening called HyperTransformer, in which features of LR-HSI and PAN are formulated as queries and keys in a transformer, respectively.
  arXiv  Detail & Related papers  (2022-03-04T18:59:08Z)
• Snapshot Hyperspectral Imaging Based on Weighted High-order Singular Value Regularization [22.5033027930853]
  Snapshot hyperspectral imaging can capture the 3D hyperspectral image (HSI) with a single 2D measurement. Existing reconstruction methods cannot fully exploit the structurally spectral-spatial nature in 3D HSI. We propose an effective high-order tensor optimization based method to boost the reconstruction fidelity for snapshot hyperspectral imaging.
  arXiv  Detail & Related papers  (2021-01-22T02:54:55Z)
• SLCRF: Subspace Learning with Conditional Random Field for Hyperspectral Image Classification [13.541897463935305]
  Subspace learning (SL) plays an important role in hyperspectral image (HSI) classification, since it can provide an effective solution to reduce the redundant information in the image pixels of HSIs. Previous works about SL aim to improve the accuracy of HSI recognition. A novel SL method that includes the probability assumption called subspace learning with conditional random field (SLCRF) is developed.
  arXiv  Detail & Related papers  (2020-10-07T02:25:32Z)
• Hyperspectral Image Super-resolution via Deep Progressive Zero-centric Residual Learning [62.52242684874278]
  Cross-modality distribution of spatial and spectral information makes the problem challenging. We propose a novel textitlightweight deep neural network-based framework, namely PZRes-Net. Our framework learns a high resolution and textitzero-centric residual image, which contains high-frequency spatial details of the scene.
  arXiv  Detail & Related papers  (2020-06-18T06:32:11Z)
• Learning Spatial-Spectral Prior for Super-Resolution of Hyperspectral Imagery [79.69449412334188]
  In this paper, we investigate how to adapt state-of-the-art residual learning based single gray/RGB image super-resolution approaches. We introduce a spatial-spectral prior network (SSPN) to fully exploit the spatial information and the correlation between the spectra of the hyperspectral data. Experimental results on some hyperspectral images demonstrate that the proposed SSPSR method enhances the details of the recovered high-resolution hyperspectral images.
  arXiv  Detail & Related papers  (2020-05-18T14:25:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.