Hierarchical Conditional Flow: A Unified Framework for Image Super-Resolution and Image Rescaling

• URL: http://arxiv.org/abs/2108.05301v1
• Date: Wed, 11 Aug 2021 16:11:01 GMT
• Title: Hierarchical Conditional Flow: A Unified Framework for Image Super-Resolution and Image Rescaling
• Authors: Jingyun Liang, Andreas Lugmayr, Kai Zhang, Martin Danelljan, Luc Van Gool, Radu Timofte
• Abstract summary: We propose a hierarchical conditional flow (HCFlow) as a unified framework for image SR and image rescaling. HCFlow learns a mapping between HR and LR image pairs by modelling the distribution of the LR image and the rest high-frequency component simultaneously. To further enhance the performance, other losses such as perceptual loss and GAN loss are combined with the commonly used negative log-likelihood loss in training.
• Score: 139.25215100378284
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Normalizing flows have recently demonstrated promising results for low-level vision tasks. For image super-resolution (SR), it learns to predict diverse photo-realistic high-resolution (HR) images from the low-resolution (LR) image rather than learning a deterministic mapping. For image rescaling, it achieves high accuracy by jointly modelling the downscaling and upscaling processes. While existing approaches employ specialized techniques for these two tasks, we set out to unify them in a single formulation. In this paper, we propose the hierarchical conditional flow (HCFlow) as a unified framework for image SR and image rescaling. More specifically, HCFlow learns a bijective mapping between HR and LR image pairs by modelling the distribution of the LR image and the rest high-frequency component simultaneously. In particular, the high-frequency component is conditional on the LR image in a hierarchical manner. To further enhance the performance, other losses such as perceptual loss and GAN loss are combined with the commonly used negative log-likelihood loss in training. Extensive experiments on general image SR, face image SR and image rescaling have demonstrated that the proposed HCFlow achieves state-of-the-art performance in terms of both quantitative metrics and visual quality.

Related papers

• Realistic Extreme Image Rescaling via Generative Latent Space Learning [51.85790402171696]
  We propose a novel framework called Latent Space Based Image Rescaling (LSBIR) for extreme image rescaling tasks. LSBIR effectively leverages powerful natural image priors learned by a pre-trained text-to-image diffusion model to generate realistic HR images. In the first stage, a pseudo-invertible encoder-decoder models the bidirectional mapping between the latent features of the HR image and the target-sized LR image. In the second stage, the reconstructed features from the first stage are refined by a pre-trained diffusion model to generate more faithful and visually pleasing details.
  arXiv  Detail & Related papers  (2024-08-17T09:51:42Z)
• Efficient Test-Time Adaptation for Super-Resolution with Second-Order Degradation and Reconstruction [62.955327005837475]
  Image super-resolution (SR) aims to learn a mapping from low-resolution (LR) to high-resolution (HR) using paired HR-LR training images. We present an efficient test-time adaptation framework for SR, named SRTTA, which is able to quickly adapt SR models to test domains with different/unknown degradation types.
  arXiv  Detail & Related papers  (2023-10-29T13:58:57Z)
• Learning Many-to-Many Mapping for Unpaired Real-World Image Super-resolution and Downscaling [60.80788144261183]
  We propose an image downscaling and SR model dubbed as SDFlow, which simultaneously learns a bidirectional many-to-many mapping between real-world LR and HR images unsupervisedly. Experimental results on real-world image SR datasets indicate that SDFlow can generate diverse realistic LR and SR images both quantitatively and qualitatively.
  arXiv  Detail & Related papers  (2023-10-08T01:48:34Z)
• Real Image Super-Resolution using GAN through modeling of LR and HR process [20.537597542144916]
  We propose a learnable adaptive sinusoidal nonlinearities incorporated in LR and SR models by directly learn degradation distributions. We demonstrate the effectiveness of our proposed approach in quantitative and qualitative experiments.
  arXiv  Detail & Related papers  (2022-10-19T09:23:37Z)
• Enhancing Image Rescaling using Dual Latent Variables in Invertible Neural Network [42.18106162158025]
  A new downscaling latent variable is introduced to model variations in the image downscaling process. It can improve image upscaling accuracy consistently without sacrificing image quality in downscaled LR images. It is also shown to be effective in enhancing other INN-based models for image restoration applications like image hiding.
  arXiv  Detail & Related papers  (2022-07-24T23:12:51Z)
• Real-World Super-Resolution of Face-Images from Surveillance Cameras [25.258587196435464]
  We propose a novel framework for generation of realistic LR/HR training pairs. Our framework estimates realistic blur kernels, noise distributions, and JPEG compression artifacts to generate LR images with similar image characteristics as the ones in the source domain. For better perceptual quality we use a Generative Adrial Network (GAN) based SR model where we have exchanged the commonly used VGG-loss [24] with LPIPS-loss [52]
  arXiv  Detail & Related papers  (2021-02-05T11:38:30Z)
• Frequency Consistent Adaptation for Real World Super Resolution [64.91914552787668]
  We propose a novel Frequency Consistent Adaptation (FCA) that ensures the frequency domain consistency when applying Super-Resolution (SR) methods to the real scene. We estimate degradation kernels from unsupervised images and generate the corresponding Low-Resolution (LR) images. Based on the domain-consistent LR-HR pairs, we train easy-implemented Convolutional Neural Network (CNN) SR models.
  arXiv  Detail & Related papers  (2020-12-18T08:25:39Z)
• SRFlow: Learning the Super-Resolution Space with Normalizing Flow [176.07982398988747]
  Super-resolution is an ill-posed problem, since it allows for multiple predictions for a given low-resolution image. We propose SRFlow: a normalizing flow based super-resolution method capable of learning the conditional distribution of the output. Our model is trained in a principled manner using a single loss, namely the negative log-likelihood.
  arXiv  Detail & Related papers  (2020-06-25T06:34:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.