Towards Bidirectional Arbitrary Image Rescaling: Joint Optimization and Cycle Idempotence

• URL: http://arxiv.org/abs/2203.00911v1
• Date: Wed, 2 Mar 2022 07:42:15 GMT
• Title: Towards Bidirectional Arbitrary Image Rescaling: Joint Optimization and Cycle Idempotence
• Authors: Zhihong Pan, Baopu Li, Dongliang He, Mingde Yao, Wenhao Wu, Tianwei Lin, Xin Li, Errui Ding
• Abstract summary: We propose a method to treat arbitrary rescaling, both upscaling and downscaling, as one unified process. The proposed model is able to learn upscaling and downscaling simultaneously and achieve bidirectional arbitrary image rescaling. It is shown to be robust in cycle idempotence test, free of severe degradations in reconstruction accuracy when the downscaling-to-upscaling cycle is applied repetitively.
• Score: 76.93002743194974
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Deep learning based single image super-resolution models have been widely studied and superb results are achieved in upscaling low-resolution images with fixed scale factor and downscaling degradation kernel. To improve real world applicability of such models, there are growing interests to develop models optimized for arbitrary upscaling factors. Our proposed method is the first to treat arbitrary rescaling, both upscaling and downscaling, as one unified process. Using joint optimization of both directions, the proposed model is able to learn upscaling and downscaling simultaneously and achieve bidirectional arbitrary image rescaling. It improves the performance of current arbitrary upscaling models by a large margin while at the same time learns to maintain visual perception quality in downscaled images. The proposed model is further shown to be robust in cycle idempotence test, free of severe degradations in reconstruction accuracy when the downscaling-to-upscaling cycle is applied repetitively. This robustness is beneficial for image rescaling in the wild when this cycle could be applied to one image for multiple times. It also performs well on tests with arbitrary large scales and asymmetric scales, even when the model is not trained with such tasks. Extensive experiments are conducted to demonstrate the superior performance of our model.

Related papers

• Fast constrained sampling in pre-trained diffusion models [77.21486516041391]
  Diffusion models have dominated the field of large, generative image models. We propose an algorithm for fast-constrained sampling in large pre-trained diffusion models.
  arXiv  Detail & Related papers  (2024-10-24T14:52:38Z)
• Perceptual-Distortion Balanced Image Super-Resolution is a Multi-Objective Optimization Problem [23.833099288826045]
  Training Single-Image Super-Resolution (SISR) models using pixel-based regression losses can achieve high distortion metrics scores. However, they often results in blurry images due to insufficient recovery of high-frequency details. We propose a novel method that incorporates Multi-Objective Optimization (MOO) into the training process of SISR models to balance perceptual quality and distortion.
  arXiv  Detail & Related papers  (2024-09-05T02:14:04Z)
• Deep Equilibrium Diffusion Restoration with Parallel Sampling [120.15039525209106]
  Diffusion model-based image restoration (IR) aims to use diffusion models to recover high-quality (HQ) images from degraded images, achieving promising performance. Most existing methods need long serial sampling chains to restore HQ images step-by-step, resulting in expensive sampling time and high computation costs. In this work, we aim to rethink the diffusion model-based IR models through a different perspective, i.e., a deep equilibrium (DEQ) fixed point system, called DeqIR.
  arXiv  Detail & Related papers  (2023-11-20T08:27:56Z)
• The Perception-Robustness Tradeoff in Deterministic Image Restoration [34.50287066865267]
  We study the behavior of deterministic methods for solving inverse problems in imaging. To approach perfect quality and perfect consistency, the Lipschitz constant of the model must grow to infinity. We demonstrate our theory on single image super-resolution algorithms, addressing both noisy and noiseless settings.
  arXiv  Detail & Related papers  (2023-11-14T18:30:34Z)
• Efficient Test-Time Adaptation for Super-Resolution with Second-Order Degradation and Reconstruction [62.955327005837475]
  Image super-resolution (SR) aims to learn a mapping from low-resolution (LR) to high-resolution (HR) using paired HR-LR training images. We present an efficient test-time adaptation framework for SR, named SRTTA, which is able to quickly adapt SR models to test domains with different/unknown degradation types.
  arXiv  Detail & Related papers  (2023-10-29T13:58:57Z)
• Effective Invertible Arbitrary Image Rescaling [77.46732646918936]
  Invertible Neural Networks (INN) are able to increase upscaling accuracy significantly by optimizing the downscaling and upscaling cycle jointly. A simple and effective invertible arbitrary rescaling network (IARN) is proposed to achieve arbitrary image rescaling by training only one model in this work. It is shown to achieve a state-of-the-art (SOTA) performance in bidirectional arbitrary rescaling without compromising perceptual quality in LR outputs.
  arXiv  Detail & Related papers  (2022-09-26T22:22:30Z)
• Enhancing Image Rescaling using Dual Latent Variables in Invertible Neural Network [42.18106162158025]
  A new downscaling latent variable is introduced to model variations in the image downscaling process. It can improve image upscaling accuracy consistently without sacrificing image quality in downscaled LR images. It is also shown to be effective in enhancing other INN-based models for image restoration applications like image hiding.
  arXiv  Detail & Related papers  (2022-07-24T23:12:51Z)
• Perceptual Image Restoration with High-Quality Priori and Degradation Learning [28.93489249639681]
  We show that our model performs well in measuring the similarity between restored and degraded images. Our simultaneous restoration and enhancement framework generalizes well to real-world complicated degradation types.
  arXiv  Detail & Related papers  (2021-03-04T13:19:50Z)
• Invertible Image Rescaling [118.2653765756915]
  We develop an Invertible Rescaling Net (IRN) to produce visually-pleasing low-resolution images. We capture the distribution of the lost information using a latent variable following a specified distribution in the downscaling process.
  arXiv  Detail & Related papers  (2020-05-12T09:55:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.