FIPER: Generalizable Factorized Fields for Joint Image Compression and Super-Resolution

• URL: http://arxiv.org/abs/2410.18083v1
• Date: Wed, 23 Oct 2024 17:59:57 GMT
• Title: FIPER: Generalizable Factorized Fields for Joint Image Compression and Super-Resolution
• Authors: Yang-Che Sun, Cheng Yu Yeo, Ernie Chu, Jun-Cheng Chen, Yu-Lun Liu,
• Abstract summary: We propose a unified representation for Super-Resolution (SR) and Image Compression, termed **Factorized Fields**, motivated by the shared principles between these two tasks. We first derive our SR model, which includes a Coefficient Backbone and Basis Swin Transformer for generalizable Factorized Fields. We then leverage the strong information-recovery capabilities of the trained SR modules as priors in the compression pipeline, improving both compression efficiency and detail reconstruction.
• Score: 12.77409981295186
• License:
• Abstract: In this work, we propose a unified representation for Super-Resolution (SR) and Image Compression, termed **Factorized Fields**, motivated by the shared principles between these two tasks. Both SISR and Image Compression require recovering and preserving fine image details--whether by enhancing resolution or reconstructing compressed data. Unlike previous methods that mainly focus on network architecture, our proposed approach utilizes a basis-coefficient decomposition to explicitly capture multi-scale visual features and structural components in images, addressing the core challenges of both tasks. We first derive our SR model, which includes a Coefficient Backbone and Basis Swin Transformer for generalizable Factorized Fields. Then, to further unify these two tasks, we leverage the strong information-recovery capabilities of the trained SR modules as priors in the compression pipeline, improving both compression efficiency and detail reconstruction. Additionally, we introduce a merged-basis compression branch that consolidates shared structures, further optimizing the compression process. Extensive experiments show that our unified representation delivers state-of-the-art performance, achieving an average relative improvement of 204.4% in PSNR over the baseline in Super-Resolution (SR) and 9.35% BD-rate reduction in Image Compression compared to the previous SOTA.

Related papers

• Self-Asymmetric Invertible Network for Compression-Aware Image Rescaling [6.861753163565238]
  In real-world applications, most images are compressed for transmission. We propose the Self-Asymmetric Invertible Network (SAIN) for compression-aware image rescaling.
  arXiv  Detail & Related papers  (2023-03-04T08:33:46Z)
• Bridging Component Learning with Degradation Modelling for Blind Image Super-Resolution [69.11604249813304]
  We propose a components decomposition and co-optimization network (CDCN) for blind SR. CDCN decomposes the input LR image into structure and detail components in feature space. We present a degradation-driven learning strategy to jointly supervise the HR image detail and structure restoration process.
  arXiv  Detail & Related papers  (2022-12-03T14:53:56Z)
• HST: Hierarchical Swin Transformer for Compressed Image Super-resolution [26.589370745694502]
  We propose the Hierarchical Swin Transformer (HST) network to restore the low-resolution compressed image. Our HST achieves the fifth place in AIM 2022 challenge on the low-quality compressed image super-resolution track, with the PSNR of 23.51dB.
  arXiv  Detail & Related papers  (2022-08-21T13:41:51Z)
• Hierarchical Similarity Learning for Aliasing Suppression Image Super-Resolution [64.15915577164894]
  A hierarchical image super-resolution network (HSRNet) is proposed to suppress the influence of aliasing. HSRNet achieves better quantitative and visual performance than other works, and remits the aliasing more effectively.
  arXiv  Detail & Related papers  (2022-06-07T14:55:32Z)
• Implicit Neural Representations for Image Compression [103.78615661013623]
  Implicit Neural Representations (INRs) have gained attention as a novel and effective representation for various data types. We propose the first comprehensive compression pipeline based on INRs including quantization, quantization-aware retraining and entropy coding. We find that our approach to source compression with INRs vastly outperforms similar prior work.
  arXiv  Detail & Related papers  (2021-12-08T13:02:53Z)
• Towards Compact CNNs via Collaborative Compression [166.86915086497433]
  We propose a Collaborative Compression scheme, which joints channel pruning and tensor decomposition to compress CNN models. We achieve 52.9% FLOPs reduction by removing 48.4% parameters on ResNet-50 with only a Top-1 accuracy drop of 0.56% on ImageNet 2012.
  arXiv  Detail & Related papers  (2021-05-24T12:07:38Z)
• Regularized Compression of MRI Data: Modular Optimization of Joint Reconstruction and Coding [2.370481325034443]
  We propose a framework for joint optimization of the MRI reconstruction and lossy compression. Our method produces compressed representations of medical images that achieve improved trade-offs between quality and bit-rate. Compared to regularization-based solutions, our optimization method provides PSNR gains between 0.5 to 1 dB at high bit-rates.
  arXiv  Detail & Related papers  (2020-10-08T15:32:52Z)
• Structure-Preserving Super Resolution with Gradient Guidance [87.79271975960764]
  Structures matter in single image super resolution (SISR) Recent studies benefiting from generative adversarial network (GAN) have promoted the development of SISR. However, there are always undesired structural distortions in the recovered images.
  arXiv  Detail & Related papers  (2020-03-29T17:26:58Z)
• Learning Better Lossless Compression Using Lossy Compression [100.50156325096611]
  We leverage the powerful lossy image compression algorithm BPG to build a lossless image compression system. We model the distribution of the residual with a convolutional neural network-based probabilistic model that is conditioned on the BPG reconstruction. Finally, the image is stored using the concatenation of the bitstreams produced by BPG and the learned residual coder.
  arXiv  Detail & Related papers  (2020-03-23T11:21:52Z)
• A GAN-based Tunable Image Compression System [13.76136694287327]
  This paper rethinks content-based compression by using Generative Adversarial Network (GAN) to reconstruct the non-important regions. A tunable compression scheme is also proposed in this paper to compress an image to any specific compression ratio without retraining the model.
  arXiv  Detail & Related papers  (2020-01-18T02:40:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.