ESC-MISR: Enhancing Spatial Correlations for Multi-Image Super-Resolution in Remote Sensing

• URL: http://arxiv.org/abs/2411.04706v1
• Date: Thu, 07 Nov 2024 13:45:04 GMT
• Title: ESC-MISR: Enhancing Spatial Correlations for Multi-Image Super-Resolution in Remote Sensing
• Authors: Zhihui Zhang, Jinhui Pang, Jianan Li, Xiaoshuai Hao,
• Abstract summary: Multi-Image Super-Resolution in Remote Sensing (MISR-RS) is a crucial yet challenging research task in the remote sensing community. We propose a novel framework named Enhancing Spatial Correlations in MISR (ESC-MISR), which fully exploits the spatial-temporal relations of multiple images for HR image reconstruction. Our method achieves 0.70dB and 0.76dB cPSNR improvements on the two bands of the PROBA-V dataset respectively.
• Score: 9.782167510476599
• License:
• Abstract: Multi-Image Super-Resolution (MISR) is a crucial yet challenging research task in the remote sensing community. In this paper, we address the challenging task of Multi-Image Super-Resolution in Remote Sensing (MISR-RS), aiming to generate a High-Resolution (HR) image from multiple Low-Resolution (LR) images obtained by satellites. Recently, the weak temporal correlations among LR images have attracted increasing attention in the MISR-RS task. However, existing MISR methods treat the LR images as sequences with strong temporal correlations, overlooking spatial correlations and imposing temporal dependencies. To address this problem, we propose a novel end-to-end framework named Enhancing Spatial Correlations in MISR (ESC-MISR), which fully exploits the spatial-temporal relations of multiple images for HR image reconstruction. Specifically, we first introduce a novel fusion module named Multi-Image Spatial Transformer (MIST), which emphasizes parts with clearer global spatial features and enhances the spatial correlations between LR images. Besides, we perform a random shuffle strategy for the sequential inputs of LR images to attenuate temporal dependencies and capture weak temporal correlations in the training stage. Compared with the state-of-the-art methods, our ESC-MISR achieves 0.70dB and 0.76dB cPSNR improvements on the two bands of the PROBA-V dataset respectively, demonstrating the superiority of our method.

Related papers

• Learning Two-factor Representation for Magnetic Resonance Image Super-resolution [1.294284364022674]
  We propose a novel method for MR image super-resolution based on two-factor representation. Specifically, we factorize intensity signals into a linear combination of learnable basis and coefficient factors. Our method achieves state-of-the-art performance, providing superior visual fidelity and robustness.
  arXiv  Detail & Related papers  (2024-09-15T13:32:24Z)
• UnmixingSR: Material-aware Network with Unsupervised Unmixing as Auxiliary Task for Hyperspectral Image Super-resolution [5.167168688234238]
  This paper proposes a component-aware hyperspectral image (HIS) super-resolution network called UnmixingSR. We use the bond between LR abundances and HR abundances to boost the stability of our method in solving SR problems. Experimental results show that unmixing process as an auxiliary task incorporated into the SR problem is feasible and rational.
  arXiv  Detail & Related papers  (2024-07-09T03:41:02Z)
• Frequency-Assisted Mamba for Remote Sensing Image Super-Resolution [49.902047563260496]
  We develop the first attempt to integrate the Vision State Space Model (Mamba) for remote sensing image (RSI) super-resolution. To achieve better SR reconstruction, building upon Mamba, we devise a Frequency-assisted Mamba framework, dubbed FMSR. Our FMSR features a multi-level fusion architecture equipped with the Frequency Selection Module (FSM), Vision State Space Module (VSSM), and Hybrid Gate Module (HGM)
  arXiv  Detail & Related papers  (2024-05-08T11:09:24Z)
• Deep 3D World Models for Multi-Image Super-Resolution Beyond Optical Flow [27.31768206943397]
  Multi-image super-resolution (MISR) allows to increase the spatial resolution of a low-resolution (LR) acquisition by combining multiple images. Our proposed model, called EpiMISR, moves away from optical flow and explicitly uses the epipolar geometry of the acquisition process.
  arXiv  Detail & Related papers  (2024-01-30T12:55:49Z)
• Real-World Image Super-Resolution by Exclusionary Dual-Learning [98.36096041099906]
  Real-world image super-resolution is a practical image restoration problem that aims to obtain high-quality images from in-the-wild input. Deep learning-based methods have achieved promising restoration quality on real-world image super-resolution datasets. We propose Real-World image Super-Resolution by Exclusionary Dual-Learning (RWSR-EDL) to address the feature diversity in perceptual- and L1-based cooperative learning.
  arXiv  Detail & Related papers  (2022-06-06T13:28:15Z)
• Memory-augmented Deep Unfolding Network for Guided Image Super-resolution [67.83489239124557]
  Guided image super-resolution (GISR) aims to obtain a high-resolution (HR) target image by enhancing the spatial resolution of a low-resolution (LR) target image under the guidance of a HR image. Previous model-based methods mainly takes the entire image as a whole, and assume the prior distribution between the HR target image and the HR guidance image. We propose a maximal a posterior (MAP) estimation model for GISR with two types of prior on the HR target image.
  arXiv  Detail & Related papers  (2022-02-12T15:37:13Z)
• IREM: High-Resolution Magnetic Resonance (MR) Image Reconstruction via Implicit Neural Representation [33.55719364798433]
  We propose a novel image reconstruction network named IREM, which is trained on multiple low-resolution (LR) MR images. IREM reduces scan time and achieves high-resolution MR imaging in terms of SNR and local image detail.
  arXiv  Detail & Related papers  (2021-06-29T05:25:43Z)
• MASA-SR: Matching Acceleration and Spatial Adaptation for Reference-Based Image Super-Resolution [74.24676600271253]
  We propose the MASA network for RefSR, where two novel modules are designed to address these problems. The proposed Match & Extraction Module significantly reduces the computational cost by a coarse-to-fine correspondence matching scheme. The Spatial Adaptation Module learns the difference of distribution between the LR and Ref images, and remaps the distribution of Ref features to that of LR features in a spatially adaptive way.
  arXiv  Detail & Related papers  (2021-06-04T07:15:32Z)
• SRDiff: Single Image Super-Resolution with Diffusion Probabilistic Models [19.17571465274627]
  Single image super-resolution (SISR) aims to reconstruct high-resolution (HR) images from the given low-resolution (LR) ones. We propose a novel single image super-resolution diffusion probabilistic model (SRDiff) SRDiff is optimized with a variant of the variational bound on the data likelihood and can provide diverse and realistic SR predictions.
  arXiv  Detail & Related papers  (2021-04-30T12:31:25Z)
• MuCAN: Multi-Correspondence Aggregation Network for Video Super-Resolution [63.02785017714131]
  Video super-resolution (VSR) aims to utilize multiple low-resolution frames to generate a high-resolution prediction for each frame. Inter- and intra-frames are the key sources for exploiting temporal and spatial information. We build an effective multi-correspondence aggregation network (MuCAN) for VSR.
  arXiv  Detail & Related papers  (2020-07-23T05:41:27Z)
• HighRes-net: Recursive Fusion for Multi-Frame Super-Resolution of Satellite Imagery [55.253395881190436]
  Multi-frame Super-Resolution (MFSR) offers a more grounded approach to the ill-posed problem. This is important for satellite monitoring of human impact on the planet. We present HighRes-net, the first deep learning approach to MFSR that learns its sub-tasks in an end-to-end fashion.
  arXiv  Detail & Related papers  (2020-02-15T22:17:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.