Band-wise Hyperspectral Image Pansharpening using CNN Model Propagation
- URL: http://arxiv.org/abs/2311.06510v1
- Date: Sat, 11 Nov 2023 08:53:54 GMT
- Title: Band-wise Hyperspectral Image Pansharpening using CNN Model Propagation
- Authors: Giuseppe Guarino, Matteo Ciotola, Gemine Vivone, Giuseppe Scarpa
- Abstract summary: We propose a new deep learning method for hyperspectral pansharpening.
It inherits a simple single-band unsupervised pansharpening model nested in a sequential band-wise adaptive scheme.
The proposed method achieves very good results on our datasets, outperforming both traditional and deep learning reference methods.
- Score: 4.246657212475299
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Hyperspectral pansharpening is receiving a growing interest since the last
few years as testified by a large number of research papers and challenges. It
consists in a pixel-level fusion between a lower-resolution hyperspectral
datacube and a higher-resolution single-band image, the panchromatic image,
with the goal of providing a hyperspectral datacube at panchromatic resolution.
Thanks to their powerful representational capabilities, deep learning models
have succeeded to provide unprecedented results on many general purpose image
processing tasks. However, when moving to domain specific problems, as in this
case, the advantages with respect to traditional model-based approaches are
much lesser clear-cut due to several contextual reasons. Scarcity of training
data, lack of ground-truth, data shape variability, are some such factors that
limit the generalization capacity of the state-of-the-art deep learning
networks for hyperspectral pansharpening. To cope with these limitations, in
this work we propose a new deep learning method which inherits a simple
single-band unsupervised pansharpening model nested in a sequential band-wise
adaptive scheme, where each band is pansharpened refining the model tuned on
the preceding one. By doing so, a simple model is propagated along the
wavelength dimension, adaptively and flexibly, with no need to have a fixed
number of spectral bands, and, with no need to dispose of large, expensive and
labeled training datasets. The proposed method achieves very good results on
our datasets, outperforming both traditional and deep learning reference
methods. The implementation of the proposed method can be found on
https://github.com/giu-guarino/R-PNN
Related papers
- Multi-Head Attention Residual Unfolded Network for Model-Based Pansharpening [2.874893537471256]
Unfolding fusion methods integrate the powerful representation capabilities of deep learning with the robustness of model-based approaches.
In this paper, we propose a model-based deep unfolded method for satellite image fusion.
Experimental results on PRISMA, Quickbird, and WorldView2 datasets demonstrate the superior performance of our method.
arXiv Detail & Related papers (2024-09-04T13:05:00Z) - Hyper-VolTran: Fast and Generalizable One-Shot Image to 3D Object
Structure via HyperNetworks [53.67497327319569]
We introduce a novel neural rendering technique to solve image-to-3D from a single view.
Our approach employs the signed distance function as the surface representation and incorporates generalizable priors through geometry-encoding volumes and HyperNetworks.
Our experiments show the advantages of our proposed approach with consistent results and rapid generation.
arXiv Detail & Related papers (2023-12-24T08:42:37Z) - Unsupervised Deep Learning-based Pansharpening with Jointly-Enhanced
Spectral and Spatial Fidelity [4.425982186154401]
We propose a new deep learning-based pansharpening model that fully exploits the potential of this approach.
The proposed model features a novel loss function that jointly promotes the spectral and spatial quality of the pansharpened data.
Experiments on a large variety of test images, performed in challenging scenarios, demonstrate that the proposed method compares favorably with the state of the art.
arXiv Detail & Related papers (2023-07-26T17:25:28Z) - Unsupervised Hyperspectral Pansharpening via Low-rank Diffusion Model [43.71116483554516]
Hyperspectral pansharpening is a process of merging a high-resolution panchromatic (PAN) image and a low-resolution hyperspectral (LRHS) image to create a single high-resolution hyperspectral (HRHS) image.
Existing Bayesian-based HS pansharpening methods require designing handcraft image prior to characterize the image features.
We propose a low-rank diffusion model for hyperspectral pansharpening by simultaneously leveraging the power of the pre-trained deep diffusion model and better generalization ability of Bayesian methods.
arXiv Detail & Related papers (2023-05-18T12:38:29Z) - Parallax-Tolerant Unsupervised Deep Image Stitching [57.76737888499145]
We propose UDIS++, a parallax-tolerant unsupervised deep image stitching technique.
First, we propose a robust and flexible warp to model the image registration from global homography to local thin-plate spline motion.
To further eliminate the parallax artifacts, we propose to composite the stitched image seamlessly by unsupervised learning for seam-driven composition masks.
arXiv Detail & Related papers (2023-02-16T10:40:55Z) - RecRecNet: Rectangling Rectified Wide-Angle Images by Thin-Plate Spline
Model and DoF-based Curriculum Learning [62.86400614141706]
We propose a new learning model, i.e., Rectangling Rectification Network (RecRecNet)
Our model can flexibly warp the source structure to the target domain and achieves an end-to-end unsupervised deformation.
Experiments show the superiority of our solution over the compared methods on both quantitative and qualitative evaluations.
arXiv Detail & Related papers (2023-01-04T15:12:57Z) - Proximal PanNet: A Model-Based Deep Network for Pansharpening [11.695233311615498]
We propose a novel deep network for pansharpening by combining the model-based methodology with the deep learning method.
We unfold the iterative algorithm into a deep network, dubbed as Proximal PanNet, by learning the proximal operators using convolutional neural networks.
Experimental results on some benchmark datasets show that our network performs better than other advanced methods both quantitatively and qualitatively.
arXiv Detail & Related papers (2022-02-12T15:49:13Z) - Learning Discriminative Shrinkage Deep Networks for Image Deconvolution [122.79108159874426]
We propose an effective non-blind deconvolution approach by learning discriminative shrinkage functions to implicitly model these terms.
Experimental results show that the proposed method performs favorably against the state-of-the-art ones in terms of efficiency and accuracy.
arXiv Detail & Related papers (2021-11-27T12:12:57Z) - Deep Gradient Projection Networks for Pan-sharpening [20.929492740317915]
This paper develops a model-based deep pan-sharpening approach.
By stacking the two blocks, a novel network, called gradient projection based pan-sharpening neural network, is constructed.
The experimental results on different kinds of satellite datasets demonstrate that the new network outperforms state-of-the-art methods both visually and quantitatively.
arXiv Detail & Related papers (2021-03-08T07:51:58Z) - Recurrent Multi-view Alignment Network for Unsupervised Surface
Registration [79.72086524370819]
Learning non-rigid registration in an end-to-end manner is challenging due to the inherent high degrees of freedom and the lack of labeled training data.
We propose to represent the non-rigid transformation with a point-wise combination of several rigid transformations.
We also introduce a differentiable loss function that measures the 3D shape similarity on the projected multi-view 2D depth images.
arXiv Detail & Related papers (2020-11-24T14:22:42Z) - Deep Unfolding Network for Image Super-Resolution [159.50726840791697]
This paper proposes an end-to-end trainable unfolding network which leverages both learning-based methods and model-based methods.
The proposed network inherits the flexibility of model-based methods to super-resolve blurry, noisy images for different scale factors via a single model.
arXiv Detail & Related papers (2020-03-23T17:55:42Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.