Zero-Shot Image Compression with Diffusion-Based Posterior Sampling

• URL: http://arxiv.org/abs/2407.09896v2
• Date: Mon, 14 Oct 2024 18:13:47 GMT
• Title: Zero-Shot Image Compression with Diffusion-Based Posterior Sampling
• Authors: Noam Elata, Tomer Michaeli, Michael Elad,
• Abstract summary: This work addresses the gap by harnessing the image prior learned by existing pre-trained diffusion models for solving the task of lossy image compression. Our method, PSC (Posterior Sampling-based Compression), utilizes zero-shot diffusion-based posterior samplers. PSC achieves competitive results compared to established methods, paving the way for further exploration of pre-trained diffusion models and posterior samplers for image compression.
• Score: 34.50287066865267
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Diffusion models dominate the field of image generation, however they have yet to make major breakthroughs in the field of image compression. Indeed, while pre-trained diffusion models have been successfully adapted to a wide variety of downstream tasks, existing work in diffusion-based image compression require task specific model training, which can be both cumbersome and limiting. This work addresses this gap by harnessing the image prior learned by existing pre-trained diffusion models for solving the task of lossy image compression. This enables the use of the wide variety of publicly-available models, and avoids the need for training or fine-tuning. Our method, PSC (Posterior Sampling-based Compression), utilizes zero-shot diffusion-based posterior samplers. It does so through a novel sequential process inspired by the active acquisition technique "Adasense" to accumulate informative measurements of the image. This strategy minimizes uncertainty in the reconstructed image and allows for construction of an image-adaptive transform coordinated between both the encoder and decoder. PSC offers a progressive compression scheme that is both practical and simple to implement. Despite minimal tuning, and a simple quantization and entropy coding, PSC achieves competitive results compared to established methods, paving the way for further exploration of pre-trained diffusion models and posterior samplers for image compression.

Related papers

• Lossy Image Compression with Foundation Diffusion Models [10.407650300093923]
  In this work we formulate the removal of quantization error as a denoising task, using diffusion to recover lost information in the transmitted image latent. Our approach allows us to perform less than 10% of the full diffusion generative process and requires no architectural changes to the diffusion model.
  arXiv  Detail & Related papers  (2024-04-12T16:23:42Z)
• Correcting Diffusion-Based Perceptual Image Compression with Privileged End-to-End Decoder [49.01721042973929]
  This paper presents a diffusion-based image compression method that employs a privileged end-to-end decoder model as correction. Experiments demonstrate the superiority of our method in both distortion and perception compared with previous perceptual compression methods.
  arXiv  Detail & Related papers  (2024-04-07T10:57:54Z)
• Unifying Generation and Compression: Ultra-low bitrate Image Coding Via Multi-stage Transformer [35.500720262253054]
  This paper introduces a novel Unified Image Generation-Compression (UIGC) paradigm, merging the processes of generation and compression. A key feature of the UIGC framework is the adoption of vector-quantized (VQ) image models for tokenization. Experiments demonstrate the superiority of the proposed UIGC framework over existing codecs in perceptual quality and human perception.
  arXiv  Detail & Related papers  (2024-03-06T14:27:02Z)
• A Training-Free Defense Framework for Robust Learned Image Compression [48.41990144764295]
  We study the robustness of learned image compression models against adversarial attacks. We present a training-free defense technique based on simple image transform functions.
  arXiv  Detail & Related papers  (2024-01-22T12:50:21Z)
• Transferable Learned Image Compression-Resistant Adversarial Perturbations [66.46470251521947]
  Adversarial attacks can readily disrupt the image classification system, revealing the vulnerability of DNN-based recognition tasks. We introduce a new pipeline that targets image classification models that utilize learned image compressors as pre-processing modules.
  arXiv  Detail & Related papers  (2024-01-06T03:03:28Z)
• Image Inpainting via Tractable Steering of Diffusion Models [54.13818673257381]
  This paper proposes to exploit the ability of Tractable Probabilistic Models (TPMs) to exactly and efficiently compute the constrained posterior. Specifically, this paper adopts a class of expressive TPMs termed Probabilistic Circuits (PCs) We show that our approach can consistently improve the overall quality and semantic coherence of inpainted images with only 10% additional computational overhead.
  arXiv  Detail & Related papers  (2023-11-28T21:14:02Z)
• Exploring Effective Mask Sampling Modeling for Neural Image Compression [171.35596121939238]
  Most existing neural image compression methods rely on side information from hyperprior or context models to eliminate spatial redundancy. Inspired by the mask sampling modeling in recent self-supervised learning methods for natural language processing and high-level vision, we propose a novel pretraining strategy for neural image compression. Our method achieves competitive performance with lower computational complexity compared to state-of-the-art image compression methods.
  arXiv  Detail & Related papers  (2023-06-09T06:50:20Z)
• Reducing The Amortization Gap of Entropy Bottleneck In End-to-End Image Compression [2.1485350418225244]
  End-to-end deep trainable models are about to exceed the performance of the traditional handcrafted compression techniques on videos and images. We propose a simple yet efficient instance-based parameterization method to reduce this amortization gap at a minor cost.
  arXiv  Detail & Related papers  (2022-09-02T11:43:45Z)
• Post-Training Quantization for Cross-Platform Learned Image Compression [15.67527732099067]
  It has been witnessed that learned image compression has outperformed conventional image coding techniques. One of the most critical issues that need to be considered is the non-deterministic calculation. We propose to solve this problem by introducing well-developed post-training quantization.
  arXiv  Detail & Related papers  (2022-02-15T15:41:12Z)
• Implicit Neural Representations for Image Compression [103.78615661013623]
  Implicit Neural Representations (INRs) have gained attention as a novel and effective representation for various data types. We propose the first comprehensive compression pipeline based on INRs including quantization, quantization-aware retraining and entropy coding. We find that our approach to source compression with INRs vastly outperforms similar prior work.
  arXiv  Detail & Related papers  (2021-12-08T13:02:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.