PGMAN: An Unsupervised Generative Multi-adversarial Network for
Pan-sharpening
- URL: http://arxiv.org/abs/2012.09054v1
- Date: Wed, 16 Dec 2020 16:21:03 GMT
- Title: PGMAN: An Unsupervised Generative Multi-adversarial Network for
Pan-sharpening
- Authors: Huanyu Zhou and Qingjie Liu and Yunhong Wang
- Abstract summary: We propose an unsupervised framework that learns directly from the full-resolution images without any preprocessing.
We use a two-stream generator to extract the modality-specific features from the PAN and MS images, respectively, and develop a dual-discriminator to preserve the spectral and spatial information of the inputs when performing fusion.
- Score: 46.84573725116611
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Pan-sharpening aims at fusing a low-resolution (LR) multi-spectral (MS) image
and a high-resolution (HR) panchromatic (PAN) image acquired by a satellite to
generate an HR MS image. Many deep learning based methods have been developed
in the past few years. However, since there are no intended HR MS images as
references for learning, almost all of the existing methods down-sample the MS
and PAN images and regard the original MS images as targets to form a
supervised setting for training. These methods may perform well on the
down-scaled images, however, they generalize poorly to the full-resolution
images. To conquer this problem, we design an unsupervised framework that is
able to learn directly from the full-resolution images without any
preprocessing. The model is built based on a novel generative multi-adversarial
network. We use a two-stream generator to extract the modality-specific
features from the PAN and MS images, respectively, and develop a
dual-discriminator to preserve the spectral and spatial information of the
inputs when performing fusion. Furthermore, a novel loss function is introduced
to facilitate training under the unsupervised setting. Experiments and
comparisons with other state-of-the-art methods on GaoFen-2 and QuickBird
images demonstrate that the proposed method can obtain much better fusion
results on the full-resolution images.
Related papers
- Variational Zero-shot Multispectral Pansharpening [43.881891055500496]
Pansharpening aims to generate a high spatial resolution multispectral image (HRMS) by fusing a low spatial resolution multispectral image (LRMS) and a panchromatic image (PAN)
Existing deep learning-based methods are unsuitable since they rely on many training pairs.
We propose a zero-shot pansharpening method by introducing a neural network into the optimization objective.
arXiv Detail & Related papers (2024-07-09T07:59:34Z) - CrossDiff: Exploring Self-Supervised Representation of Pansharpening via
Cross-Predictive Diffusion Model [42.39485365164292]
Fusion of a panchromatic (PAN) image and corresponding multispectral (MS) image is also known as pansharpening.
Due to the absence of high-resolution MS images, available deep-learning-based methods usually follow the paradigm of training at reduced resolution and testing at both reduced and full resolution.
We propose to explore the self-supervised representation of pansharpening by designing a cross-predictive diffusion model, named CrossDiff.
arXiv Detail & Related papers (2024-01-10T13:32:47Z) - Learning from Multi-Perception Features for Real-Word Image
Super-resolution [87.71135803794519]
We propose a novel SR method called MPF-Net that leverages multiple perceptual features of input images.
Our method incorporates a Multi-Perception Feature Extraction (MPFE) module to extract diverse perceptual information.
We also introduce a contrastive regularization term (CR) that improves the model's learning capability.
arXiv Detail & Related papers (2023-05-26T07:35:49Z) - PC-GANs: Progressive Compensation Generative Adversarial Networks for
Pan-sharpening [50.943080184828524]
We propose a novel two-step model for pan-sharpening that sharpens the MS image through the progressive compensation of the spatial and spectral information.
The whole model is composed of triple GANs, and based on the specific architecture, a joint compensation loss function is designed to enable the triple GANs to be trained simultaneously.
arXiv Detail & Related papers (2022-07-29T03:09:21Z) - Memory-augmented Deep Unfolding Network for Guided Image
Super-resolution [67.83489239124557]
Guided image super-resolution (GISR) aims to obtain a high-resolution (HR) target image by enhancing the spatial resolution of a low-resolution (LR) target image under the guidance of a HR image.
Previous model-based methods mainly takes the entire image as a whole, and assume the prior distribution between the HR target image and the HR guidance image.
We propose a maximal a posterior (MAP) estimation model for GISR with two types of prior on the HR target image.
arXiv Detail & Related papers (2022-02-12T15:37:13Z) - LDP-Net: An Unsupervised Pansharpening Network Based on Learnable
Degradation Processes [18.139096037746672]
We propose a novel unsupervised network based on learnable degradation processes, dubbed as LDP-Net.
A reblurring block and a graying block are designed to learn the corresponding degradation processes, respectively.
Experiments on Worldview2 and Worldview3 images demonstrate that our proposed LDP-Net can fuse PAN and LRMS images effectively without the help of HRMS samples.
arXiv Detail & Related papers (2021-11-24T13:21:22Z) - Unsupervised Cycle-consistent Generative Adversarial Networks for
Pan-sharpening [41.68141846006704]
We propose an unsupervised generative adversarial framework that learns from the full-scale images without the ground truths to alleviate this problem.
We extract the modality-specific features from the PAN and MS images with a two-stream generator, perform fusion in the feature domain, and then reconstruct the pan-sharpened images.
Results demonstrate that the proposed method can greatly improve the pan-sharpening performance on the full-scale images.
arXiv Detail & Related papers (2021-09-20T09:43:24Z) - Fast and High-Quality Blind Multi-Spectral Image Pansharpening [48.68143888901669]
We propose a fast approach to blind pansharpening and achieve state-of-the-art image reconstruction quality.
To achieve fast blind pansharpening, we decouple the solution of the blur kernel and of the HRMS image.
Our algorithm outperforms state-of-the-art model-based counterparts in terms of both computational time and reconstruction quality.
arXiv Detail & Related papers (2021-03-17T23:12:14Z) - PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of
Generative Models [77.32079593577821]
PULSE (Photo Upsampling via Latent Space Exploration) generates high-resolution, realistic images at resolutions previously unseen in the literature.
Our method outperforms state-of-the-art methods in perceptual quality at higher resolutions and scale factors than previously possible.
arXiv Detail & Related papers (2020-03-08T16:44:31Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.