Self-Supervised Learning for Real-World Super-Resolution from Dual Zoomed Observations

• URL: http://arxiv.org/abs/2203.01325v1
• Date: Wed, 2 Mar 2022 13:30:56 GMT
• Title: Self-Supervised Learning for Real-World Super-Resolution from Dual Zoomed Observations
• Authors: Zhilu Zhang, Ruohao Wang, Hongzhi Zhang, Yunjin Chen, Wangmeng Zuo
• Abstract summary: We present a novel self-supervised learning approach for real-world RefSR from observations at dual camera zooms (SelfDZSR) For the first issue, the more zoomed (telephoto) image can be naturally leveraged as the reference to guide the SR of the lesser zoomed (short-focus) image. For the second issue, SelfDZSR learns a deep network to obtain the SR result of short-focal image and with the same resolution as the telephoto image.
• Score: 66.09210030518686
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we consider two challenging issues in reference-based super-resolution (RefSR), (i) how to choose a proper reference image, and (ii) how to learn real-world RefSR in a self-supervised manner. Particularly, we present a novel self-supervised learning approach for real-world image SR from observations at dual camera zooms (SelfDZSR). For the first issue, the more zoomed (telephoto) image can be naturally leveraged as the reference to guide the SR of the lesser zoomed (short-focus) image. For the second issue, SelfDZSR learns a deep network to obtain the SR result of short-focal image and with the same resolution as the telephoto image. For this purpose, we take the telephoto image instead of an additional high-resolution image as the supervision information and select a patch from it as the reference to super-resolve the corresponding short-focus image patch. To mitigate the effect of various misalignment between the short-focus low-resolution (LR) image and telephoto ground-truth (GT) image, we design a degradation model and map the GT to a pseudo-LR image aligned with GT. Then the pseudo-LR and LR image can be fed into the proposed adaptive spatial transformer networks (AdaSTN) to deform the LR features. During testing, SelfDZSR can be directly deployed to super-solve the whole short-focus image with the reference of telephoto image. Experiments show that our method achieves better quantitative and qualitative performance against state-of-the-arts. The code and pre-trained models will be publicly available.

Related papers

• ClearSR: Latent Low-Resolution Image Embeddings Help Diffusion-Based Real-World Super Resolution Models See Clearer [68.72454974431749]
  We present ClearSR, a new method that can better take advantage of latent low-resolution image (LR) embeddings for diffusion-based real-world image super-resolution (Real-ISR) Our model can achieve better performance across multiple metrics on several test sets and generate more consistent SR results with LR images than existing methods.
  arXiv  Detail & Related papers  (2024-10-18T08:35:57Z)
• Self-Supervised Learning for Real-World Super-Resolution from Dual and Multiple Zoomed Observations [61.448005005426666]
  We consider two challenging issues in reference-based super-resolution (RefSR) for smartphone. We propose a novel self-supervised learning approach for real-world RefSR from observations at dual and multiple camera zooms.
  arXiv  Detail & Related papers  (2024-05-03T15:20:30Z)
• Dual-Camera Super-Resolution with Aligned Attention Modules [56.54073689003269]
  We present a novel approach to reference-based super-resolution (RefSR) with the focus on dual-camera super-resolution (DCSR) Our proposed method generalizes the standard patch-based feature matching with spatial alignment operations. To bridge the domain gaps between real-world images and the training images, we propose a self-supervised domain adaptation strategy.
  arXiv  Detail & Related papers  (2021-09-03T07:17:31Z)
• Benefiting from Bicubically Down-Sampled Images for Learning Real-World Image Super-Resolution [22.339751911637077]
  We propose to handle real-world SR by splitting this ill-posed problem into two comparatively more well-posed steps. First, we train a network to transform real LR images to the space of bicubically downsampled images in a supervised manner. Second, we take a generic SR network trained on bicubically downsampled images to super-resolve the transformed LR image.
  arXiv  Detail & Related papers  (2020-07-06T20:27:58Z)
• Closed-loop Matters: Dual Regression Networks for Single Image Super-Resolution [73.86924594746884]
  Deep neural networks have exhibited promising performance in image super-resolution. These networks learn a nonlinear mapping function from low-resolution (LR) images to high-resolution (HR) images. We propose a dual regression scheme by introducing an additional constraint on LR data to reduce the space of the possible functions.
  arXiv  Detail & Related papers  (2020-03-16T04:23:42Z)
• Learning to Zoom-in via Learning to Zoom-out: Real-world Super-resolution by Generating and Adapting Degradation [91.40265983636839]
  We propose a framework to learn SR from an arbitrary set of unpaired LR and HR images. We minimize the discrepancy between the generated data and real data while learning a degradation adaptive SR network. The proposed unpaired method achieves state-of-the-art SR results on real-world images, even in the datasets that favor the paired-learning methods more.
  arXiv  Detail & Related papers  (2020-01-08T05:17:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.