Enhancing Image Rescaling using Dual Latent Variables in Invertible Neural Network

• URL: http://arxiv.org/abs/2207.11844v1
• Date: Sun, 24 Jul 2022 23:12:51 GMT
• Title: Enhancing Image Rescaling using Dual Latent Variables in Invertible Neural Network
• Authors: Min Zhang, Zhihong Pan, Xin Zhou, C.-C. Jay Kuo
• Abstract summary: A new downscaling latent variable is introduced to model variations in the image downscaling process. It can improve image upscaling accuracy consistently without sacrificing image quality in downscaled LR images. It is also shown to be effective in enhancing other INN-based models for image restoration applications like image hiding.
• Score: 42.18106162158025
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Normalizing flow models have been used successfully for generative image super-resolution (SR) by approximating complex distribution of natural images to simple tractable distribution in latent space through Invertible Neural Networks (INN). These models can generate multiple realistic SR images from one low-resolution (LR) input using randomly sampled points in the latent space, simulating the ill-posed nature of image upscaling where multiple high-resolution (HR) images correspond to the same LR. Lately, the invertible process in INN has also been used successfully by bidirectional image rescaling models like IRN and HCFlow for joint optimization of downscaling and inverse upscaling, resulting in significant improvements in upscaled image quality. While they are optimized for image downscaling too, the ill-posed nature of image downscaling, where one HR image could be downsized to multiple LR images depending on different interpolation kernels and resampling methods, is not considered. A new downscaling latent variable, in addition to the original one representing uncertainties in image upscaling, is introduced to model variations in the image downscaling process. This dual latent variable enhancement is applicable to different image rescaling models and it is shown in extensive experiments that it can improve image upscaling accuracy consistently without sacrificing image quality in downscaled LR images. It is also shown to be effective in enhancing other INN-based models for image restoration applications like image hiding.

Related papers

• Realistic Extreme Image Rescaling via Generative Latent Space Learning [51.85790402171696]
  We propose a novel framework called Latent Space Based Image Rescaling (LSBIR) for extreme image rescaling tasks. LSBIR effectively leverages powerful natural image priors learned by a pre-trained text-to-image diffusion model to generate realistic HR images. In the first stage, a pseudo-invertible encoder-decoder models the bidirectional mapping between the latent features of the HR image and the target-sized LR image. In the second stage, the reconstructed features from the first stage are refined by a pre-trained diffusion model to generate more faithful and visually pleasing details.
  arXiv  Detail & Related papers  (2024-08-17T09:51:42Z)
• Raising The Limit Of Image Rescaling Using Auxiliary Encoding [7.9700865143145485]
  Recently, image rescaling models like IRN utilize the bidirectional nature of INN to push the performance limit of image upscaling. We propose auxiliary encoding modules to further push the limit of image rescaling performance.
  arXiv  Detail & Related papers  (2023-03-12T20:49:07Z)
• Super-resolution Reconstruction of Single Image for Latent features [8.857209365343646]
  Single-image super-resolution (SISR) typically focuses on restoring various degraded low-resolution (LR) images to a single high-resolution (HR) image. It is often challenging for models to simultaneously maintain high quality and rapid sampling while preserving diversity in details and texture features. This challenge can lead to issues such as model collapse, lack of rich details and texture features in the reconstructed HR images, and excessive time consumption for model sampling.
  arXiv  Detail & Related papers  (2022-11-16T09:37:07Z)
• Real Image Super-Resolution using GAN through modeling of LR and HR process [20.537597542144916]
  We propose a learnable adaptive sinusoidal nonlinearities incorporated in LR and SR models by directly learn degradation distributions. We demonstrate the effectiveness of our proposed approach in quantitative and qualitative experiments.
  arXiv  Detail & Related papers  (2022-10-19T09:23:37Z)
• Effective Invertible Arbitrary Image Rescaling [77.46732646918936]
  Invertible Neural Networks (INN) are able to increase upscaling accuracy significantly by optimizing the downscaling and upscaling cycle jointly. A simple and effective invertible arbitrary rescaling network (IARN) is proposed to achieve arbitrary image rescaling by training only one model in this work. It is shown to achieve a state-of-the-art (SOTA) performance in bidirectional arbitrary rescaling without compromising perceptual quality in LR outputs.
  arXiv  Detail & Related papers  (2022-09-26T22:22:30Z)
• Hierarchical Conditional Flow: A Unified Framework for Image Super-Resolution and Image Rescaling [139.25215100378284]
  We propose a hierarchical conditional flow (HCFlow) as a unified framework for image SR and image rescaling. HCFlow learns a mapping between HR and LR image pairs by modelling the distribution of the LR image and the rest high-frequency component simultaneously. To further enhance the performance, other losses such as perceptual loss and GAN loss are combined with the commonly used negative log-likelihood loss in training.
  arXiv  Detail & Related papers  (2021-08-11T16:11:01Z)
• Designing a Practical Degradation Model for Deep Blind Image Super-Resolution [134.9023380383406]
  Single image super-resolution (SISR) methods would not perform well if the assumed degradation model deviates from those in real images. This paper proposes to design a more complex but practical degradation model that consists of randomly shuffled blur, downsampling and noise degradations.
  arXiv  Detail & Related papers  (2021-03-25T17:40:53Z)
• Invertible Image Rescaling [118.2653765756915]
  We develop an Invertible Rescaling Net (IRN) to produce visually-pleasing low-resolution images. We capture the distribution of the lost information using a latent variable following a specified distribution in the downscaling process.
  arXiv  Detail & Related papers  (2020-05-12T09:55:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.