Learning Dual-Level Deformable Implicit Representation for Real-World Scale Arbitrary Super-Resolution

• URL: http://arxiv.org/abs/2403.10925v1
• Date: Sat, 16 Mar 2024 13:44:42 GMT
• Title: Learning Dual-Level Deformable Implicit Representation for Real-World Scale Arbitrary Super-Resolution
• Authors: Zhiheng Li, Muheng Li, Jixuan Fan, Lei Chen, Yansong Tang, Jie Zhou, Jiwen Lu,
• Abstract summary: We build a new real-world super-resolution benchmark with both integer and non-integer scaling factors for the training and evaluation of real-world scale arbitrary super-resolution. Specifically, we design the appearance embedding and deformation field to handle both image-level and pixel-level deformations caused by real-world degradations. Our trained model achieves state-of-the-art performance on the RealArbiSR and RealSR benchmarks for real-world scale arbitrary super-resolution.
• Score: 81.74583887661794
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Scale arbitrary super-resolution based on implicit image function gains increasing popularity since it can better represent the visual world in a continuous manner. However, existing scale arbitrary works are trained and evaluated on simulated datasets, where low-resolution images are generated from their ground truths by the simplest bicubic downsampling. These models exhibit limited generalization to real-world scenarios due to the greater complexity of real-world degradations. To address this issue, we build a RealArbiSR dataset, a new real-world super-resolution benchmark with both integer and non-integer scaling factors for the training and evaluation of real-world scale arbitrary super-resolution. Moreover, we propose a Dual-level Deformable Implicit Representation (DDIR) to solve real-world scale arbitrary super-resolution. Specifically, we design the appearance embedding and deformation field to handle both image-level and pixel-level deformations caused by real-world degradations. The appearance embedding models the characteristics of low-resolution inputs to deal with photometric variations at different scales, and the pixel-based deformation field learns RGB differences which result from the deviations between the real-world and simulated degradations at arbitrary coordinates. Extensive experiments show our trained model achieves state-of-the-art performance on the RealArbiSR and RealSR benchmarks for real-world scale arbitrary super-resolution. Our dataset as well as source code will be publicly available.

Related papers

• Enhanced Super-Resolution Training via Mimicked Alignment for Real-World Scenes [51.92255321684027]
  We propose a novel plug-and-play module designed to mitigate misalignment issues by aligning LR inputs with HR images during training. Specifically, our approach involves mimicking a novel LR sample that aligns with HR while preserving the characteristics of the original LR samples. We comprehensively evaluate our method on synthetic and real-world datasets, demonstrating its effectiveness across a spectrum of SR models.
  arXiv  Detail & Related papers  (2024-10-07T18:18:54Z)
• Pairwise Distance Distillation for Unsupervised Real-World Image Super-Resolution [38.79439380482431]
  Real-world super-resolution (RWSR) faces unknown degradations in the low-resolution inputs, all the while lacking paired training data. Existing methods approach this problem by learning blind general models through complex synthetic augmentations on training inputs. We introduce a novel pairwise distance distillation framework to address the unsupervised RWSR for a targeted real-world degradation.
  arXiv  Detail & Related papers  (2024-07-10T01:46:40Z)
• Towards Realistic Data Generation for Real-World Super-Resolution [58.88039242455039]
  RealDGen is an unsupervised learning data generation framework designed for real-world super-resolution. We develop content and degradation extraction strategies, which are integrated into a novel content-degradation decoupled diffusion model. Experiments demonstrate that RealDGen excels in generating large-scale, high-quality paired data that mirrors real-world degradations.
  arXiv  Detail & Related papers  (2024-06-11T13:34:57Z)
• Towards Real-World Burst Image Super-Resolution: Benchmark and Method [93.73429028287038]
  In this paper, we establish a large-scale real-world burst super-resolution dataset, i.e., RealBSR, to explore the faithful reconstruction of image details from multiple frames. We also introduce a Federated Burst Affinity network (FBAnet) to investigate non-trivial pixel-wise displacement among images under real-world image degradation.
  arXiv  Detail & Related papers  (2023-09-09T14:11:37Z)
• Exploiting Digital Surface Models for Inferring Super-Resolution for Remotely Sensed Images [2.3204178451683264]
  This paper introduces a novel approach for forcing an SRR model to output realistic remote sensing images. Instead of relying on feature-space similarities as a perceptual loss, the model considers pixel-level information inferred from the normalized Digital Surface Model (nDSM) of the image. Based on visual inspection, the inferred super-resolution images exhibit particularly superior quality.
  arXiv  Detail & Related papers  (2022-05-09T06:02:50Z)
• Any-resolution Training for High-resolution Image Synthesis [55.19874755679901]
  Generative models operate at fixed resolution, even though natural images come in a variety of sizes. We argue that every pixel matters and create datasets with variable-size images, collected at their native resolutions. We introduce continuous-scale training, a process that samples patches at random scales to train a new generator with variable output resolutions.
  arXiv  Detail & Related papers  (2022-04-14T17:59:31Z)
• Scale-arbitrary Invertible Image Downscaling [17.67415618760949]
  We propose a scale-Arbitrary Invertible image Downscaling Network (AIDN) to downscale HR images with arbitrary scale factors. Our AIDN achieves top performance for invertible downscaling with both arbitrary integer and non-integer scale factors.
  arXiv  Detail & Related papers  (2022-01-29T12:27:52Z)
• Benefiting from Bicubically Down-Sampled Images for Learning Real-World Image Super-Resolution [22.339751911637077]
  We propose to handle real-world SR by splitting this ill-posed problem into two comparatively more well-posed steps. First, we train a network to transform real LR images to the space of bicubically downsampled images in a supervised manner. Second, we take a generic SR network trained on bicubically downsampled images to super-resolve the transformed LR image.
  arXiv  Detail & Related papers  (2020-07-06T20:27:58Z)
• PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Models [77.32079593577821]
  PULSE (Photo Upsampling via Latent Space Exploration) generates high-resolution, realistic images at resolutions previously unseen in the literature. Our method outperforms state-of-the-art methods in perceptual quality at higher resolutions and scale factors than previously possible.
  arXiv  Detail & Related papers  (2020-03-08T16:44:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.