Self-Asymmetric Invertible Network for Compression-Aware Image Rescaling
- URL: http://arxiv.org/abs/2303.02353v1
- Date: Sat, 4 Mar 2023 08:33:46 GMT
- Title: Self-Asymmetric Invertible Network for Compression-Aware Image Rescaling
- Authors: Jinhai Yang, Mengxi Guo, Shijie Zhao, Junlin Li, Li Zhang
- Abstract summary: In real-world applications, most images are compressed for transmission.
We propose the Self-Asymmetric Invertible Network (SAIN) for compression-aware image rescaling.
- Score: 6.861753163565238
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: High-resolution (HR) images are usually downscaled to low-resolution (LR)
ones for better display and afterward upscaled back to the original size to
recover details. Recent work in image rescaling formulates downscaling and
upscaling as a unified task and learns a bijective mapping between HR and LR
via invertible networks. However, in real-world applications (e.g., social
media), most images are compressed for transmission. Lossy compression will
lead to irreversible information loss on LR images, hence damaging the inverse
upscaling procedure and degrading the reconstruction accuracy. In this paper,
we propose the Self-Asymmetric Invertible Network (SAIN) for compression-aware
image rescaling. To tackle the distribution shift, we first develop an
end-to-end asymmetric framework with two separate bijective mappings for
high-quality and compressed LR images, respectively. Then, based on empirical
analysis of this framework, we model the distribution of the lost information
(including downscaling and compression) using isotropic Gaussian mixtures and
propose the Enhanced Invertible Block to derive high-quality/compressed LR
images in one forward pass. Besides, we design a set of losses to regularize
the learned LR images and enhance the invertibility. Extensive experiments
demonstrate the consistent improvements of SAIN across various image rescaling
datasets in terms of both quantitative and qualitative evaluation under
standard image compression formats (i.e., JPEG and WebP).
Related papers
- FIPER: Generalizable Factorized Fields for Joint Image Compression and Super-Resolution [12.77409981295186]
We propose a unified representation for Super-Resolution (SR) and Image Compression, termed **Factorized Fields**, motivated by the shared principles between these two tasks.
We first derive our SR model, which includes a Coefficient Backbone and Basis Swin Transformer for generalizable Factorized Fields.
We then leverage the strong information-recovery capabilities of the trained SR modules as priors in the compression pipeline, improving both compression efficiency and detail reconstruction.
arXiv Detail & Related papers (2024-10-23T17:59:57Z) - Realistic Extreme Image Rescaling via Generative Latent Space Learning [51.85790402171696]
We propose a novel framework called Latent Space Based Image Rescaling (LSBIR) for extreme image rescaling tasks.
LSBIR effectively leverages powerful natural image priors learned by a pre-trained text-to-image diffusion model to generate realistic HR images.
In the first stage, a pseudo-invertible encoder-decoder models the bidirectional mapping between the latent features of the HR image and the target-sized LR image.
In the second stage, the reconstructed features from the first stage are refined by a pre-trained diffusion model to generate more faithful and visually pleasing details.
arXiv Detail & Related papers (2024-08-17T09:51:42Z) - Invertible Rescaling Network and Its Extensions [118.72015270085535]
In this work, we propose a novel invertible framework to model the bidirectional degradation and restoration from a new perspective.
We develop invertible models to generate valid degraded images and transform the distribution of lost contents.
Then restoration is made tractable by applying the inverse transformation on the generated degraded image together with a randomly-drawn latent variable.
arXiv Detail & Related papers (2022-10-09T06:58:58Z) - Enhancing Image Rescaling using Dual Latent Variables in Invertible
Neural Network [42.18106162158025]
A new downscaling latent variable is introduced to model variations in the image downscaling process.
It can improve image upscaling accuracy consistently without sacrificing image quality in downscaled LR images.
It is also shown to be effective in enhancing other INN-based models for image restoration applications like image hiding.
arXiv Detail & Related papers (2022-07-24T23:12:51Z) - Implicit Neural Representations for Image Compression [103.78615661013623]
Implicit Neural Representations (INRs) have gained attention as a novel and effective representation for various data types.
We propose the first comprehensive compression pipeline based on INRs including quantization, quantization-aware retraining and entropy coding.
We find that our approach to source compression with INRs vastly outperforms similar prior work.
arXiv Detail & Related papers (2021-12-08T13:02:53Z) - Hierarchical Conditional Flow: A Unified Framework for Image
Super-Resolution and Image Rescaling [139.25215100378284]
We propose a hierarchical conditional flow (HCFlow) as a unified framework for image SR and image rescaling.
HCFlow learns a mapping between HR and LR image pairs by modelling the distribution of the LR image and the rest high-frequency component simultaneously.
To further enhance the performance, other losses such as perceptual loss and GAN loss are combined with the commonly used negative log-likelihood loss in training.
arXiv Detail & Related papers (2021-08-11T16:11:01Z) - Super-Resolution of Real-World Faces [3.4376560669160394]
Real low-resolution (LR) face images contain degradations which are too varied and complex to be captured by known downsampling kernels.
In this paper, we propose a two module super-resolution network where the feature extractor module extracts robust features from the LR image.
We train a degradation GAN to convert bicubically downsampled clean images to real degraded images, and interpolate between the obtained degraded LR image and its clean LR counterpart.
arXiv Detail & Related papers (2020-11-04T17:25:54Z) - Modeling Lost Information in Lossy Image Compression [72.69327382643549]
Lossy image compression is one of the most commonly used operators for digital images.
We propose a novel invertible framework called Invertible Lossy Compression (ILC) to largely mitigate the information loss problem.
arXiv Detail & Related papers (2020-06-22T04:04:56Z) - Invertible Image Rescaling [118.2653765756915]
We develop an Invertible Rescaling Net (IRN) to produce visually-pleasing low-resolution images.
We capture the distribution of the lost information using a latent variable following a specified distribution in the downscaling process.
arXiv Detail & Related papers (2020-05-12T09:55:53Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.