Deep dual stream residual network with contextual attention for pansharpening of remote sensing images

• URL: http://arxiv.org/abs/2207.12004v1
• Date: Mon, 25 Jul 2022 09:28:11 GMT
• Title: Deep dual stream residual network with contextual attention for pansharpening of remote sensing images
• Authors: Syeda Roshana Ali, Anis Ur Rahman, Muhammad Shahzad
• Abstract summary: We present a novel dual attention-based two-stream network. It starts with feature extraction using two separate networks for both images, an encoder with attention mechanism to recalibrate the extracted features. This is followed by fusion of the features forming a compact representation fed into an image reconstruction network to produce a pansharpened image.
• Score: 2.210012031884757
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Pansharpening enhances spatial details of high spectral resolution multispectral images using features of high spatial resolution panchromatic image. There are a number of traditional pansharpening approaches but producing an image exhibiting high spectral and spatial fidelity is still an open problem. Recently, deep learning has been used to produce promising pansharpened images; however, most of these approaches apply similar treatment to both multispectral and panchromatic images by using the same network for feature extraction. In this work, we present present a novel dual attention-based two-stream network. It starts with feature extraction using two separate networks for both images, an encoder with attention mechanism to recalibrate the extracted features. This is followed by fusion of the features forming a compact representation fed into an image reconstruction network to produce a pansharpened image. The experimental results on the Pl\'{e}iades dataset using standard quantitative evaluation metrics and visual inspection demonstrates that the proposed approach performs better than other approaches in terms of pansharpened image quality.

Related papers

• PanBench: Towards High-Resolution and High-Performance Pansharpening [16.16122045172545]
  Pansharpening involves integrating low-resolution multispectral images with high-resolution panchromatic images to synthesize an image that is both high-resolution and retains multispectral information. This paper introduces PanBench, a high-resolution multi-scene dataset containing all mainstream satellites. To achieve high-fidelity synthesis, we propose a Cascaded Multiscale Fusion Network (CMFNet) for Pansharpening.
  arXiv  Detail & Related papers  (2023-11-20T10:57:23Z)
• PC-GANs: Progressive Compensation Generative Adversarial Networks for Pan-sharpening [50.943080184828524]
  We propose a novel two-step model for pan-sharpening that sharpens the MS image through the progressive compensation of the spatial and spectral information. The whole model is composed of triple GANs, and based on the specific architecture, a joint compensation loss function is designed to enable the triple GANs to be trained simultaneously.
  arXiv  Detail & Related papers  (2022-07-29T03:09:21Z)
• Single Stage Virtual Try-on via Deformable Attention Flows [51.70606454288168]
  Virtual try-on aims to generate a photo-realistic fitting result given an in-shop garment and a reference person image. We develop a novel Deformable Attention Flow (DAFlow) which applies the deformable attention scheme to multi-flow estimation. Our proposed method achieves state-of-the-art performance both qualitatively and quantitatively.
  arXiv  Detail & Related papers  (2022-07-19T10:01:31Z)
• Cross-View Panorama Image Synthesis [68.35351563852335]
  PanoGAN is a novel adversarial feedback GAN framework named. PanoGAN enables high-quality panorama image generation with more convincing details than state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-03-22T15:59:44Z)
• Image Inpainting with Edge-guided Learnable Bidirectional Attention Maps [85.67745220834718]
  We present an edge-guided learnable bidirectional attention map (Edge-LBAM) for improving image inpainting of irregular holes. Our Edge-LBAM method contains dual procedures,including structure-aware mask-updating guided by predict edges. Extensive experiments show that our Edge-LBAM is effective in generating coherent image structures and preventing color discrepancy and blurriness.
  arXiv  Detail & Related papers  (2021-04-25T07:25:16Z)
• Deep Gradient Projection Networks for Pan-sharpening [20.929492740317915]
  This paper develops a model-based deep pan-sharpening approach. By stacking the two blocks, a novel network, called gradient projection based pan-sharpening neural network, is constructed. The experimental results on different kinds of satellite datasets demonstrate that the new network outperforms state-of-the-art methods both visually and quantitatively.
  arXiv  Detail & Related papers  (2021-03-08T07:51:58Z)
• Learning deep multiresolution representations for pansharpening [4.469255274378329]
  This paper proposes a pyramid based deep fusion framework that preserves spectral and spatial characteristics at different scales. Experiments suggest that the proposed architecture outperforms state of the art pansharpening models.
  arXiv  Detail & Related papers  (2021-02-16T19:41:57Z)
• Split then Refine: Stacked Attention-guided ResUNets for Blind Single Image Visible Watermark Removal [69.92767260794628]
  Previous watermark removal methods require to gain the watermark location from users or train a multi-task network to recover the background indiscriminately. We propose a novel two-stage framework with a stacked attention-guided ResUNets to simulate the process of detection, removal and refinement. We extensively evaluate our algorithm over four different datasets under various settings and the experiments show that our approach outperforms other state-of-the-art methods by a large margin.
  arXiv  Detail & Related papers  (2020-12-13T09:05:37Z)
• Deep 3D Capture: Geometry and Reflectance from Sparse Multi-View Images [59.906948203578544]
  We introduce a novel learning-based method to reconstruct the high-quality geometry and complex, spatially-varying BRDF of an arbitrary object. We first estimate per-view depth maps using a deep multi-view stereo network. These depth maps are used to coarsely align the different views. We propose a novel multi-view reflectance estimation network architecture.
  arXiv  Detail & Related papers  (2020-03-27T21:28:54Z)
• Pyramidal Edge-maps and Attention based Guided Thermal Super-resolution [28.798966778371145]
  Guided super-resolution (GSR) of thermal images using visible range images is challenging because of the difference in the spectral-range between the images. We propose a novel algorithm for GSR based on pyramidal edge-maps extracted from the visible image. Our model outperforms the state-of-the-art GSR methods, both quantitatively and qualitatively.
  arXiv  Detail & Related papers  (2020-03-13T12:11:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.