Efficient Test-Time Adaptation for Super-Resolution with Second-Order Degradation and Reconstruction

• URL: http://arxiv.org/abs/2310.19011v1
• Date: Sun, 29 Oct 2023 13:58:57 GMT
• Title: Efficient Test-Time Adaptation for Super-Resolution with Second-Order Degradation and Reconstruction
• Authors: Zeshuai Deng, Zhuokun Chen, Shuaicheng Niu, Thomas H. Li, Bohan Zhuang, Mingkui Tan
• Abstract summary: Image super-resolution (SR) aims to learn a mapping from low-resolution (LR) to high-resolution (HR) using paired HR-LR training images. We present an efficient test-time adaptation framework for SR, named SRTTA, which is able to quickly adapt SR models to test domains with different/unknown degradation types.
• Score: 62.955327005837475
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Image super-resolution (SR) aims to learn a mapping from low-resolution (LR) to high-resolution (HR) using paired HR-LR training images. Conventional SR methods typically gather the paired training data by synthesizing LR images from HR images using a predetermined degradation model, e.g., Bicubic down-sampling. However, the realistic degradation type of test images may mismatch with the training-time degradation type due to the dynamic changes of the real-world scenarios, resulting in inferior-quality SR images. To address this, existing methods attempt to estimate the degradation model and train an image-specific model, which, however, is quite time-consuming and impracticable to handle rapidly changing domain shifts. Moreover, these methods largely concentrate on the estimation of one degradation type (e.g., blur degradation), overlooking other degradation types like noise and JPEG in real-world test-time scenarios, thus limiting their practicality. To tackle these problems, we present an efficient test-time adaptation framework for SR, named SRTTA, which is able to quickly adapt SR models to test domains with different/unknown degradation types. Specifically, we design a second-order degradation scheme to construct paired data based on the degradation type of the test image, which is predicted by a pre-trained degradation classifier. Then, we adapt the SR model by implementing feature-level reconstruction learning from the initial test image to its second-order degraded counterparts, which helps the SR model generate plausible HR images. Extensive experiments are conducted on newly synthesized corrupted DIV2K datasets with 8 different degradations and several real-world datasets, demonstrating that our SRTTA framework achieves an impressive improvement over existing methods with satisfying speed. The source code is available at https://github.com/DengZeshuai/SRTTA.

Related papers

• Learning Many-to-Many Mapping for Unpaired Real-World Image Super-resolution and Downscaling [60.80788144261183]
  We propose an image downscaling and SR model dubbed as SDFlow, which simultaneously learns a bidirectional many-to-many mapping between real-world LR and HR images unsupervisedly. Experimental results on real-world image SR datasets indicate that SDFlow can generate diverse realistic LR and SR images both quantitatively and qualitatively.
  arXiv  Detail & Related papers  (2023-10-08T01:48:34Z)
• DR2: Diffusion-based Robust Degradation Remover for Blind Face Restoration [66.01846902242355]
  Blind face restoration usually synthesizes degraded low-quality data with a pre-defined degradation model for training. It is expensive and infeasible to include every type of degradation to cover real-world cases in the training data. We propose Robust Degradation Remover (DR2) to first transform the degraded image to a coarse but degradation-invariant prediction, then employ an enhancement module to restore the coarse prediction to a high-quality image.
  arXiv  Detail & Related papers  (2023-03-13T06:05:18Z)
• Blind Super-Resolution for Remote Sensing Images via Conditional Stochastic Normalizing Flows [14.882417028542855]
  We propose a novel blind SR framework based on the normalizing flow (BlindSRSNF) to address the above problems. BlindSRSNF learns the conditional probability distribution over the high-resolution image space given a low-resolution (LR) image by explicitly optimizing the variational bound on the likelihood. We show that the proposed algorithm can obtain SR results with excellent visual perception quality on both simulated LR and real-world RSIs.
  arXiv  Detail & Related papers  (2022-10-14T12:37:32Z)
• Toward Real-world Image Super-resolution via Hardware-based Adaptive Degradation Models [3.9037347042028254]
  Most single image super-resolution (SR) methods are developed on synthetic low-resolution (LR) and high-resolution (HR) image pairs. We propose a novel supervised method to simulate an unknown degradation process with the inclusion of prior hardware knowledge. Experiments on the real-world datasets validate that our degradation model can estimate LR images more accurately than the predetermined degradation operation.
  arXiv  Detail & Related papers  (2021-10-20T19:53:48Z)
• Hierarchical Conditional Flow: A Unified Framework for Image Super-Resolution and Image Rescaling [139.25215100378284]
  We propose a hierarchical conditional flow (HCFlow) as a unified framework for image SR and image rescaling. HCFlow learns a mapping between HR and LR image pairs by modelling the distribution of the LR image and the rest high-frequency component simultaneously. To further enhance the performance, other losses such as perceptual loss and GAN loss are combined with the commonly used negative log-likelihood loss in training.
  arXiv  Detail & Related papers  (2021-08-11T16:11:01Z)
• Unsupervised Degradation Representation Learning for Blind Super-Resolution [27.788488575616032]
  CNN-based super-resolution (SR) methods suffer a severe performance drop when the real degradation is different from their assumption. We propose an unsupervised degradation representation learning scheme for blind SR without explicit degradation estimation. Our network achieves state-of-the-art performance for the blind SR task.
  arXiv  Detail & Related papers  (2021-04-01T11:57:42Z)
• Frequency Consistent Adaptation for Real World Super Resolution [64.91914552787668]
  We propose a novel Frequency Consistent Adaptation (FCA) that ensures the frequency domain consistency when applying Super-Resolution (SR) methods to the real scene. We estimate degradation kernels from unsupervised images and generate the corresponding Low-Resolution (LR) images. Based on the domain-consistent LR-HR pairs, we train easy-implemented Convolutional Neural Network (CNN) SR models.
  arXiv  Detail & Related papers  (2020-12-18T08:25:39Z)
• Joint Generative Learning and Super-Resolution For Real-World Camera-Screen Degradation [6.14297871633911]
  In real-world single image super-resolution (SISR) task, the low-resolution image suffers more complicated degradations. In this paper, we focus on the camera-screen degradation and build a real-world dataset (Cam-ScreenSR) We propose a joint two-stage model. Firstly, the downsampling degradation GAN(DD-GAN) is trained to model the degradation and produces more various of LR images. Then the dual residual channel attention network (DuRCAN) learns to recover the SR image.
  arXiv  Detail & Related papers  (2020-08-01T07:10:13Z)
• Characteristic Regularisation for Super-Resolving Face Images [81.84939112201377]
  Existing facial image super-resolution (SR) methods focus mostly on improving artificially down-sampled low-resolution (LR) imagery. Previous unsupervised domain adaptation (UDA) methods address this issue by training a model using unpaired genuine LR and HR data. This renders the model overstretched with two tasks: consistifying the visual characteristics and enhancing the image resolution. We formulate a method that joins the advantages of conventional SR and UDA models.
  arXiv  Detail & Related papers  (2019-12-30T16:27:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.