FIPER: Generalizable Factorized Fields for Joint Image Compression and Super-Resolution

• URL: http://arxiv.org/abs/2410.18083v1
• Date: Wed, 23 Oct 2024 17:59:57 GMT
• Title: FIPER: Generalizable Factorized Fields for Joint Image Compression and Super-Resolution
• Authors: Yang-Che Sun, Cheng Yu Yeo, Ernie Chu, Jun-Cheng Chen, Yu-Lun Liu,
• Abstract summary: We propose a unified representation for Super-Resolution (SR) and Image Compression, termed **Factorized Fields**, motivated by the shared principles between these two tasks. We first derive our SR model, which includes a Coefficient Backbone and Basis Swin Transformer for generalizable Factorized Fields. We then leverage the strong information-recovery capabilities of the trained SR modules as priors in the compression pipeline, improving both compression efficiency and detail reconstruction.
• Score: 12.77409981295186
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this work, we propose a unified representation for Super-Resolution (SR) and Image Compression, termed **Factorized Fields**, motivated by the shared principles between these two tasks. Both SISR and Image Compression require recovering and preserving fine image details--whether by enhancing resolution or reconstructing compressed data. Unlike previous methods that mainly focus on network architecture, our proposed approach utilizes a basis-coefficient decomposition to explicitly capture multi-scale visual features and structural components in images, addressing the core challenges of both tasks. We first derive our SR model, which includes a Coefficient Backbone and Basis Swin Transformer for generalizable Factorized Fields. Then, to further unify these two tasks, we leverage the strong information-recovery capabilities of the trained SR modules as priors in the compression pipeline, improving both compression efficiency and detail reconstruction. Additionally, we introduce a merged-basis compression branch that consolidates shared structures, further optimizing the compression process. Extensive experiments show that our unified representation delivers state-of-the-art performance, achieving an average relative improvement of 204.4% in PSNR over the baseline in Super-Resolution (SR) and 9.35% BD-rate reduction in Image Compression compared to the previous SOTA.

Related papers

• Content-decoupled Contrastive Learning-based Implicit Degradation Modeling for Blind Image Super-Resolution [33.16889233975723]
  Implicit degradation modeling-based blind super-resolution (SR) has attracted more increasing attention in the community. We propose a new Content-decoupled Contrastive Learning-based blind image super-resolution (CdCL) framework.
  arXiv  Detail & Related papers  (2024-08-10T04:51:43Z)
• Efficient Visual State Space Model for Image Deblurring [83.57239834238035]
  Convolutional neural networks (CNNs) and Vision Transformers (ViTs) have achieved excellent performance in image restoration. We propose a simple yet effective visual state space model (EVSSM) for image deblurring.
  arXiv  Detail & Related papers  (2024-05-23T09:13:36Z)
• Efficient Model Agnostic Approach for Implicit Neural Representation Based Arbitrary-Scale Image Super-Resolution [5.704360536038803]
  Single image super-resolution (SISR) has experienced significant advancements, primarily driven by deep convolutional networks. Traditional networks are limited to upscaling images to a fixed scale, leading to the utilization of implicit neural functions for generating arbitrarily scaled images. We introduce a novel and efficient framework, the Mixture of Experts Implicit Super-Resolution (MoEISR), which enables super-resolution at arbitrary scales.
  arXiv  Detail & Related papers  (2023-11-20T05:34:36Z)
• Self-Asymmetric Invertible Network for Compression-Aware Image Rescaling [6.861753163565238]
  In real-world applications, most images are compressed for transmission. We propose the Self-Asymmetric Invertible Network (SAIN) for compression-aware image rescaling.
  arXiv  Detail & Related papers  (2023-03-04T08:33:46Z)
• Bridging Component Learning with Degradation Modelling for Blind Image Super-Resolution [69.11604249813304]
  We propose a components decomposition and co-optimization network (CDCN) for blind SR. CDCN decomposes the input LR image into structure and detail components in feature space. We present a degradation-driven learning strategy to jointly supervise the HR image detail and structure restoration process.
  arXiv  Detail & Related papers  (2022-12-03T14:53:56Z)
• HST: Hierarchical Swin Transformer for Compressed Image Super-resolution [26.589370745694502]
  We propose the Hierarchical Swin Transformer (HST) network to restore the low-resolution compressed image. Our HST achieves the fifth place in AIM 2022 challenge on the low-quality compressed image super-resolution track, with the PSNR of 23.51dB.
  arXiv  Detail & Related papers  (2022-08-21T13:41:51Z)
• Rank-Enhanced Low-Dimensional Convolution Set for Hyperspectral Image Denoising [50.039949798156826]
  This paper tackles the challenging problem of hyperspectral (HS) image denoising. We propose rank-enhanced low-dimensional convolution set (Re-ConvSet) We then incorporate Re-ConvSet into the widely-used U-Net architecture to construct an HS image denoising method.
  arXiv  Detail & Related papers  (2022-07-09T13:35:12Z)
• Joint Super-Resolution and Inverse Tone-Mapping: A Feature Decomposition Aggregation Network and A New Benchmark [0.0]
  We propose a lightweight Feature Decomposition Aggregation Network (FDAN) to exploit the potential power of decomposition mechanism. In particular, we design a Feature Decomposition Block (FDB) to achieve learnable separation of detail and base feature maps. We also collect a large-scale dataset for joint SR-ITM, i.e., SRITM-4K, which provides versatile scenarios for robust model training and evaluation.
  arXiv  Detail & Related papers  (2022-07-07T15:16:36Z)
• Hierarchical Similarity Learning for Aliasing Suppression Image Super-Resolution [64.15915577164894]
  A hierarchical image super-resolution network (HSRNet) is proposed to suppress the influence of aliasing. HSRNet achieves better quantitative and visual performance than other works, and remits the aliasing more effectively.
  arXiv  Detail & Related papers  (2022-06-07T14:55:32Z)
• Dual Perceptual Loss for Single Image Super-Resolution Using ESRGAN [13.335546116599494]
  This paper proposes a method called Dual Perceptual Loss (DP Loss) to replace the original perceptual loss to solve the problem of single image super-resolution reconstruction. Due to the complementary property between the VGG features and the ResNet features, the proposed DP Loss considers the advantages of learning two features simultaneously. The qualitative and quantitative analysis on benchmark datasets demonstrates the superiority of our proposed method over state-of-the-art super-resolution methods.
  arXiv  Detail & Related papers  (2022-01-17T12:42:56Z)
• Implicit Neural Representations for Image Compression [103.78615661013623]
  Implicit Neural Representations (INRs) have gained attention as a novel and effective representation for various data types. We propose the first comprehensive compression pipeline based on INRs including quantization, quantization-aware retraining and entropy coding. We find that our approach to source compression with INRs vastly outperforms similar prior work.
  arXiv  Detail & Related papers  (2021-12-08T13:02:53Z)
• Towards Compact CNNs via Collaborative Compression [166.86915086497433]
  We propose a Collaborative Compression scheme, which joints channel pruning and tensor decomposition to compress CNN models. We achieve 52.9% FLOPs reduction by removing 48.4% parameters on ResNet-50 with only a Top-1 accuracy drop of 0.56% on ImageNet 2012.
  arXiv  Detail & Related papers  (2021-05-24T12:07:38Z)
• Regularized Compression of MRI Data: Modular Optimization of Joint Reconstruction and Coding [2.370481325034443]
  We propose a framework for joint optimization of the MRI reconstruction and lossy compression. Our method produces compressed representations of medical images that achieve improved trade-offs between quality and bit-rate. Compared to regularization-based solutions, our optimization method provides PSNR gains between 0.5 to 1 dB at high bit-rates.
  arXiv  Detail & Related papers  (2020-10-08T15:32:52Z)
• Structure-Preserving Super Resolution with Gradient Guidance [87.79271975960764]
  Structures matter in single image super resolution (SISR) Recent studies benefiting from generative adversarial network (GAN) have promoted the development of SISR. However, there are always undesired structural distortions in the recovered images.
  arXiv  Detail & Related papers  (2020-03-29T17:26:58Z)
• Learning Better Lossless Compression Using Lossy Compression [100.50156325096611]
  We leverage the powerful lossy image compression algorithm BPG to build a lossless image compression system. We model the distribution of the residual with a convolutional neural network-based probabilistic model that is conditioned on the BPG reconstruction. Finally, the image is stored using the concatenation of the bitstreams produced by BPG and the learned residual coder.
  arXiv  Detail & Related papers  (2020-03-23T11:21:52Z)
• Learning Enriched Features for Real Image Restoration and Enhancement [166.17296369600774]
  convolutional neural networks (CNNs) have achieved dramatic improvements over conventional approaches for image restoration task. We present a novel architecture with the collective goals of maintaining spatially-precise high-resolution representations through the entire network. Our approach learns an enriched set of features that combines contextual information from multiple scales, while simultaneously preserving the high-resolution spatial details.
  arXiv  Detail & Related papers  (2020-03-15T11:04:30Z)
• Gated Fusion Network for Degraded Image Super Resolution [78.67168802945069]
  We propose a dual-branch convolutional neural network to extract base features and recovered features separately. By decomposing the feature extraction step into two task-independent streams, the dual-branch model can facilitate the training process.
  arXiv  Detail & Related papers  (2020-03-02T13:28:32Z)
• A GAN-based Tunable Image Compression System [13.76136694287327]
  This paper rethinks content-based compression by using Generative Adversarial Network (GAN) to reconstruct the non-important regions. A tunable compression scheme is also proposed in this paper to compress an image to any specific compression ratio without retraining the model.
  arXiv  Detail & Related papers  (2020-01-18T02:40:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.