Good, Cheap, and Fast: Overfitted Image Compression with Wasserstein Distortion

• URL: http://arxiv.org/abs/2412.00505v1
• Date: Sat, 30 Nov 2024 15:05:01 GMT
• Title: Good, Cheap, and Fast: Overfitted Image Compression with Wasserstein Distortion
• Authors: Jona Ballé, Luca Versari, Emilien Dupont, Hyunjik Kim, Matthias Bauer,
• Abstract summary: We show that by focusing on modeling visual perception rather than the data distribution, we can achieve a good trade-off between visual quality and bit rate. We do this by optimizing C3, an overfitted image, for Wasserstein Distortion (WD) and evaluating the image reconstructions with a human rater study.
• Score: 13.196774986841469
• License:
• Abstract: Inspired by the success of generative image models, recent work on learned image compression increasingly focuses on better probabilistic models of the natural image distribution, leading to excellent image quality. This, however, comes at the expense of a computational complexity that is several orders of magnitude higher than today's commercial codecs, and thus prohibitive for most practical applications. With this paper, we demonstrate that by focusing on modeling visual perception rather than the data distribution, we can achieve a very good trade-off between visual quality and bit rate similar to "generative" compression models such as HiFiC, while requiring less than 1% of the multiply-accumulate operations (MACs) for decompression. We do this by optimizing C3, an overfitted image codec, for Wasserstein Distortion (WD), and evaluating the image reconstructions with a human rater study. The study also reveals that WD outperforms other perceptual quality metrics such as LPIPS, DISTS, and MS-SSIM, both as an optimization objective and as a predictor of human ratings, achieving over 94% Pearson correlation with Elo scores.

Related papers

• Predicting Satisfied User and Machine Ratio for Compressed Images: A Unified Approach [58.71009078356928]
  We create a deep learning-based model to predict Satisfied User Ratio (SUR) and Satisfied Machine Ratio (SMR) of compressed images simultaneously. Experimental results indicate that the proposed model significantly outperforms state-of-the-art SUR and SMR prediction methods.
  arXiv  Detail & Related papers  (2024-12-23T11:09:30Z)
• Robustly overfitting latents for flexible neural image compression [1.7041035606170198]
  State-of-the-art neural image compression models learn to encode an image into a quantized latent representation that can be efficiently sent to the decoder. These models have proven successful in practice, but they lead to sub-optimal results due to imperfect optimization and limitations in the encoder and decoder capacity. Recent work shows how to use Gumbel annealing (SGA) to refine the latents of pre-trained neural image compression models. We show how our method improves the overall compression performance in terms of the R-D trade-off, compared to its predecessors.
  arXiv  Detail & Related papers  (2024-01-31T12:32:17Z)
• Semantic Ensemble Loss and Latent Refinement for High-Fidelity Neural Image Compression [58.618625678054826]
  This study presents an enhanced neural compression method designed for optimal visual fidelity. We have trained our model with a sophisticated semantic ensemble loss, integrating Charbonnier loss, perceptual loss, style loss, and a non-binary adversarial loss. Our empirical findings demonstrate that this approach significantly improves the statistical fidelity of neural image compression.
  arXiv  Detail & Related papers  (2024-01-25T08:11:27Z)
• Machine Perception-Driven Image Compression: A Layered Generative Approach [32.23554195427311]
  layered generative image compression model is proposed to achieve high human vision-oriented image reconstructed quality. Task-agnostic learning-based compression model is proposed, which effectively supports various compressed domain-based analytical tasks. Joint optimization schedule is adopted to acquire best balance point among compression ratio, reconstructed image quality, and downstream perception performance.
  arXiv  Detail & Related papers  (2023-04-14T02:12:38Z)
• Improving Statistical Fidelity for Neural Image Compression with Implicit Local Likelihood Models [31.308949268401047]
  Lossy image compression aims to represent images in as few bits as possible while maintaining fidelity to the original. We introduce a non-binary discriminator that is conditioned on quantized local image representations obtained via VQ-VAE autoencoders.
  arXiv  Detail & Related papers  (2023-01-26T15:55:43Z)
• High-Fidelity Variable-Rate Image Compression via Invertible Activation Transformation [24.379052026260034]
  We propose the Invertible Activation Transformation (IAT) module to tackle the issue of high-fidelity fine variable-rate image compression. IAT and QLevel together give the image compression model the ability of fine variable-rate control while better maintaining the image fidelity. Our method outperforms the state-of-the-art variable-rate image compression method by a large margin, especially after multiple re-encodings.
  arXiv  Detail & Related papers  (2022-09-12T07:14:07Z)
• Estimating the Resize Parameter in End-to-end Learned Image Compression [50.20567320015102]
  We describe a search-free resizing framework that can further improve the rate-distortion tradeoff of recent learned image compression models. Our results show that our new resizing parameter estimation framework can provide Bjontegaard-Delta rate (BD-rate) improvement of about 10% against leading perceptual quality engines.
  arXiv  Detail & Related papers  (2022-04-26T01:35:02Z)
• Implicit Neural Representations for Image Compression [103.78615661013623]
  Implicit Neural Representations (INRs) have gained attention as a novel and effective representation for various data types. We propose the first comprehensive compression pipeline based on INRs including quantization, quantization-aware retraining and entropy coding. We find that our approach to source compression with INRs vastly outperforms similar prior work.
  arXiv  Detail & Related papers  (2021-12-08T13:02:53Z)
• Variable-Rate Deep Image Compression through Spatially-Adaptive Feature Transform [58.60004238261117]
  We propose a versatile deep image compression network based on Spatial Feature Transform (SFT arXiv:1804.02815) Our model covers a wide range of compression rates using a single model, which is controlled by arbitrary pixel-wise quality maps. The proposed framework allows us to perform task-aware image compressions for various tasks.
  arXiv  Detail & Related papers  (2021-08-21T17:30:06Z)
• Early Exit or Not: Resource-Efficient Blind Quality Enhancement for Compressed Images [54.40852143927333]
  Lossy image compression is pervasively conducted to save communication bandwidth, resulting in undesirable compression artifacts. We propose a resource-efficient blind quality enhancement (RBQE) approach for compressed images. Our approach can automatically decide to terminate or continue enhancement according to the assessed quality of enhanced images.
  arXiv  Detail & Related papers  (2020-06-30T07:38:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.