PanFlowNet: A Flow-Based Deep Network for Pan-sharpening

• URL: http://arxiv.org/abs/2305.07774v2
• Date: Tue, 16 May 2023 14:46:08 GMT
• Title: PanFlowNet: A Flow-Based Deep Network for Pan-sharpening
• Authors: Gang Yang, Xiangyong Cao, Wenzhe Xiao, Man Zhou, Aiping Liu, Xun chen, and Deyu Meng
• Abstract summary: Pan-sharpening aims to generate a high-resolution multispectral (HRMS) image by integrating the spectral information of a low-resolution multispectral (LRMS) image with the texture details of a high-resolution panchromatic (PAN) image. Existing deep learning-based methods recover only one HRMS image from the LRMS image and PAN image using a deterministic mapping. We propose a flow-based pan-sharpening network (PanFlowNet) to directly learn the conditional distribution of HRMS image given LRMS image and PAN image.
• Score: 41.9419544446451
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Pan-sharpening aims to generate a high-resolution multispectral (HRMS) image by integrating the spectral information of a low-resolution multispectral (LRMS) image with the texture details of a high-resolution panchromatic (PAN) image. It essentially inherits the ill-posed nature of the super-resolution (SR) task that diverse HRMS images can degrade into an LRMS image. However, existing deep learning-based methods recover only one HRMS image from the LRMS image and PAN image using a deterministic mapping, thus ignoring the diversity of the HRMS image. In this paper, to alleviate this ill-posed issue, we propose a flow-based pan-sharpening network (PanFlowNet) to directly learn the conditional distribution of HRMS image given LRMS image and PAN image instead of learning a deterministic mapping. Specifically, we first transform this unknown conditional distribution into a given Gaussian distribution by an invertible network, and the conditional distribution can thus be explicitly defined. Then, we design an invertible Conditional Affine Coupling Block (CACB) and further build the architecture of PanFlowNet by stacking a series of CACBs. Finally, the PanFlowNet is trained by maximizing the log-likelihood of the conditional distribution given a training set and can then be used to predict diverse HRMS images. The experimental results verify that the proposed PanFlowNet can generate various HRMS images given an LRMS image and a PAN image. Additionally, the experimental results on different kinds of satellite datasets also demonstrate the superiority of our PanFlowNet compared with other state-of-the-art methods both visually and quantitatively.

Related papers

• Multi-Head Attention Residual Unfolded Network for Model-Based Pansharpening [2.874893537471256]
  Unfolding fusion methods integrate the powerful representation capabilities of deep learning with the robustness of model-based approaches. In this paper, we propose a model-based deep unfolded method for satellite image fusion. Experimental results on PRISMA, Quickbird, and WorldView2 datasets demonstrate the superior performance of our method.
  arXiv  Detail & Related papers  (2024-09-04T13:05:00Z)
• Variational Zero-shot Multispectral Pansharpening [43.881891055500496]
  Pansharpening aims to generate a high spatial resolution multispectral image (HRMS) by fusing a low spatial resolution multispectral image (LRMS) and a panchromatic image (PAN) Existing deep learning-based methods are unsuitable since they rely on many training pairs. We propose a zero-shot pansharpening method by introducing a neural network into the optimization objective.
  arXiv  Detail & Related papers  (2024-07-09T07:59:34Z)
• PC-GANs: Progressive Compensation Generative Adversarial Networks for Pan-sharpening [50.943080184828524]
  We propose a novel two-step model for pan-sharpening that sharpens the MS image through the progressive compensation of the spatial and spectral information. The whole model is composed of triple GANs, and based on the specific architecture, a joint compensation loss function is designed to enable the triple GANs to be trained simultaneously.
  arXiv  Detail & Related papers  (2022-07-29T03:09:21Z)
• LDP-Net: An Unsupervised Pansharpening Network Based on Learnable Degradation Processes [18.139096037746672]
  We propose a novel unsupervised network based on learnable degradation processes, dubbed as LDP-Net. A reblurring block and a graying block are designed to learn the corresponding degradation processes, respectively. Experiments on Worldview2 and Worldview3 images demonstrate that our proposed LDP-Net can fuse PAN and LRMS images effectively without the help of HRMS samples.
  arXiv  Detail & Related papers  (2021-11-24T13:21:22Z)
• Hierarchical Conditional Flow: A Unified Framework for Image Super-Resolution and Image Rescaling [139.25215100378284]
  We propose a hierarchical conditional flow (HCFlow) as a unified framework for image SR and image rescaling. HCFlow learns a mapping between HR and LR image pairs by modelling the distribution of the LR image and the rest high-frequency component simultaneously. To further enhance the performance, other losses such as perceptual loss and GAN loss are combined with the commonly used negative log-likelihood loss in training.
  arXiv  Detail & Related papers  (2021-08-11T16:11:01Z)
• Fast and High-Quality Blind Multi-Spectral Image Pansharpening [48.68143888901669]
  We propose a fast approach to blind pansharpening and achieve state-of-the-art image reconstruction quality. To achieve fast blind pansharpening, we decouple the solution of the blur kernel and of the HRMS image. Our algorithm outperforms state-of-the-art model-based counterparts in terms of both computational time and reconstruction quality.
  arXiv  Detail & Related papers  (2021-03-17T23:12:14Z)
• PGMAN: An Unsupervised Generative Multi-adversarial Network for Pan-sharpening [46.84573725116611]
  We propose an unsupervised framework that learns directly from the full-resolution images without any preprocessing. We use a two-stream generator to extract the modality-specific features from the PAN and MS images, respectively, and develop a dual-discriminator to preserve the spectral and spatial information of the inputs when performing fusion.
  arXiv  Detail & Related papers  (2020-12-16T16:21:03Z)
• SRFlow: Learning the Super-Resolution Space with Normalizing Flow [176.07982398988747]
  Super-resolution is an ill-posed problem, since it allows for multiple predictions for a given low-resolution image. We propose SRFlow: a normalizing flow based super-resolution method capable of learning the conditional distribution of the output. Our model is trained in a principled manner using a single loss, namely the negative log-likelihood.
  arXiv  Detail & Related papers  (2020-06-25T06:34:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.