PSC: Posterior Sampling-Based Compression

• URL: http://arxiv.org/abs/2407.09896v3
• Date: Wed, 05 Feb 2025 14:33:13 GMT
• Title: PSC: Posterior Sampling-Based Compression
• Authors: Noam Elata, Tomer Michaeli, Michael Elad,
• Abstract summary: Posterior Sampling-based Compression (PSC) is a zero-shot compression method that leverages a pre-trained diffusion model as its sole neural network component. PSC constructs a transform that is adaptive to the image. We demonstrate that PSC's performance is comparable to established training-based methods in terms of rate, distortion, and perceptual quality.
• Score: 34.50287066865267
• License:
• Abstract: Diffusion models have transformed the landscape of image generation and now show remarkable potential for image compression. Most of the recent diffusion-based compression methods require training and are tailored for a specific bit-rate. In this work, we propose Posterior Sampling-based Compression (PSC) - a zero-shot compression method that leverages a pre-trained diffusion model as its sole neural network component, thus enabling the use of diverse, publicly available models without additional training. Our approach is inspired by transform coding methods, which encode the image in some pre-chosen transform domain. However, PSC constructs a transform that is adaptive to the image. This is done by employing a zero-shot diffusion-based posterior sampler so as to progressively construct the rows of the transform matrix. Each new chunk of rows is chosen to reduce the uncertainty about the image given the quantized measurements collected thus far. Importantly, the same adaptive scheme can be replicated at the decoder, thus avoiding the need to encode the transform itself. We demonstrate that even with basic quantization and entropy coding, PSC's performance is comparable to established training-based methods in terms of rate, distortion, and perceptual quality. This is while providing greater flexibility, allowing to choose at inference time any desired rate or distortion.

Related papers

• CALLIC: Content Adaptive Learning for Lossless Image Compression [64.47244912937204]
  CALLIC sets a new state-of-the-art (SOTA) for learned lossless image compression. We propose a content-aware autoregressive self-attention mechanism by leveraging convolutional gating operations. During encoding, we decompose pre-trained layers, including depth-wise convolutions, using low-rank matrices and then adapt the incremental weights on testing image by Rate-guided Progressive Fine-Tuning (RPFT) RPFT fine-tunes with gradually increasing patches that are sorted in descending order by estimated entropy, optimizing learning process and reducing adaptation time.
  arXiv  Detail & Related papers  (2024-12-23T10:41:18Z)
• Controllable Distortion-Perception Tradeoff Through Latent Diffusion for Neural Image Compression [30.293252608423742]
  Neural image compression often faces a challenging trade-off among rate, distortion and perception. We propose a novel approach that simultaneously addresses both aspects for a fixed neural image. We can achieve more than 150% improvement in LPIPS-BDRate without sacrificing more than 1 dB in PSNR.
  arXiv  Detail & Related papers  (2024-12-16T02:09:32Z)
• Correcting Diffusion-Based Perceptual Image Compression with Privileged End-to-End Decoder [49.01721042973929]
  This paper presents a diffusion-based image compression method that employs a privileged end-to-end decoder model as correction. Experiments demonstrate the superiority of our method in both distortion and perception compared with previous perceptual compression methods.
  arXiv  Detail & Related papers  (2024-04-07T10:57:54Z)
• Progressive Learning with Visual Prompt Tuning for Variable-Rate Image Compression [60.689646881479064]
  We propose a progressive learning paradigm for transformer-based variable-rate image compression. Inspired by visual prompt tuning, we use LPM to extract prompts for input images and hidden features at the encoder side and decoder side, respectively. Our model outperforms all current variable image methods in terms of rate-distortion performance and approaches the state-of-the-art fixed image compression methods trained from scratch.
  arXiv  Detail & Related papers  (2023-11-23T08:29:32Z)
• Multiscale Augmented Normalizing Flows for Image Compression [17.441496966834933]
  We present a novel concept, which adapts the hierarchical latent space for augmented normalizing flows, an invertible latent variable model. Our best performing model achieved average rate savings of more than 7% over comparable single-scale models.
  arXiv  Detail & Related papers  (2023-05-09T13:42:43Z)
• Lossy Image Compression with Conditional Diffusion Models [25.158390422252097]
  This paper outlines an end-to-end optimized lossy image compression framework using diffusion generative models. In contrast to VAE-based neural compression, where the (mean) decoder is a deterministic neural network, our decoder is a conditional diffusion model. Our approach yields stronger reported FID scores than the GAN-based model, while also yielding competitive performance with VAE-based models in several distortion metrics.
  arXiv  Detail & Related papers  (2022-09-14T21:53:27Z)
• Reducing The Amortization Gap of Entropy Bottleneck In End-to-End Image Compression [2.1485350418225244]
  End-to-end deep trainable models are about to exceed the performance of the traditional handcrafted compression techniques on videos and images. We propose a simple yet efficient instance-based parameterization method to reduce this amortization gap at a minor cost.
  arXiv  Detail & Related papers  (2022-09-02T11:43:45Z)
• Neural Data-Dependent Transform for Learned Image Compression [72.86505042102155]
  We build a neural data-dependent transform and introduce a continuous online mode decision mechanism to jointly optimize the coding efficiency for each individual image. The experimental results show the effectiveness of the proposed neural-syntax design and the continuous online mode decision mechanism.
  arXiv  Detail & Related papers  (2022-03-09T14:56:48Z)
• Post-Training Quantization for Cross-Platform Learned Image Compression [15.67527732099067]
  It has been witnessed that learned image compression has outperformed conventional image coding techniques. One of the most critical issues that need to be considered is the non-deterministic calculation. We propose to solve this problem by introducing well-developed post-training quantization.
  arXiv  Detail & Related papers  (2022-02-15T15:41:12Z)
• Towards End-to-End Image Compression and Analysis with Transformers [99.50111380056043]
  We propose an end-to-end image compression and analysis model with Transformers, targeting to the cloud-based image classification application. We aim to redesign the Vision Transformer (ViT) model to perform image classification from the compressed features and facilitate image compression with the long-term information from the Transformer. Experimental results demonstrate the effectiveness of the proposed model in both the image compression and the classification tasks.
  arXiv  Detail & Related papers  (2021-12-17T03:28:14Z)
• Modeling Lost Information in Lossy Image Compression [72.69327382643549]
  Lossy image compression is one of the most commonly used operators for digital images. We propose a novel invertible framework called Invertible Lossy Compression (ILC) to largely mitigate the information loss problem.
  arXiv  Detail & Related papers  (2020-06-22T04:04:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.