Higher fidelity perceptual image and video compression with a latent conditioned residual denoising diffusion model

• URL: http://arxiv.org/abs/2505.13152v1
• Date: Mon, 19 May 2025 14:13:14 GMT
• Title: Higher fidelity perceptual image and video compression with a latent conditioned residual denoising diffusion model
• Authors: Jonas Brenig, Radu Timofte,
• Abstract summary: We propose a hybrid compression scheme optimized for perceptual quality, extending the approach of the CDC model with a decoder network.<n>We achieve up to +2dB PSNR fidelity improvements while maintaining comparable LPIPS and FID perceptual scores when compared with CDC.
• Score: 55.2480439325792
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Denoising diffusion models achieved impressive results on several image generation tasks often outperforming GAN based models. Recently, the generative capabilities of diffusion models have been employed for perceptual image compression, such as in CDC. A major drawback of these diffusion-based methods is that, while producing impressive perceptual quality images they are dropping in fidelity/increasing the distortion to the original uncompressed images when compared with other traditional or learned image compression schemes aiming for fidelity. In this paper, we propose a hybrid compression scheme optimized for perceptual quality, extending the approach of the CDC model with a decoder network in order to reduce the impact on distortion metrics such as PSNR. After using the decoder network to generate an initial image, optimized for distortion, the latent conditioned diffusion model refines the reconstruction for perceptual quality by predicting the residual. On standard benchmarks, we achieve up to +2dB PSNR fidelity improvements while maintaining comparable LPIPS and FID perceptual scores when compared with CDC. Additionally, the approach is easily extensible to video compression, where we achieve similar results.

Related papers

• Generative Image Compression by Estimating Gradients of the Rate-variable Feature Distribution [37.60572296105984]
  We propose a novel diffusion-based generative modeling framework tailored for generative image compression.<n>A reverse neural network is trained to reconstruct images by reversing the compression process directly.<n>This approach achieves smooth rate adjustment and photo-realistic reconstructions with only a minimal number of sampling steps.
  arXiv  Detail & Related papers  (2025-05-27T10:18:24Z)
• Ultra Lowrate Image Compression with Semantic Residual Coding and Compression-aware Diffusion [28.61304513668606]
  ResULIC is a residual-guided ultra lowrate image compression system.<n>It incorporates residual signals into both semantic retrieval and the diffusion-based generation process.<n>It achieves superior objective and subjective performance compared to state-of-the-art diffusion-based methods.
  arXiv  Detail & Related papers  (2025-05-13T06:51:23Z)
• SING: Semantic Image Communications using Null-Space and INN-Guided Diffusion Models [52.40011613324083]
  Joint source-channel coding systems (DeepJSCC) have recently demonstrated remarkable performance in wireless image transmission.<n>Existing methods focus on minimizing distortion between the transmitted image and the reconstructed version at the receiver, often overlooking perceptual quality.<n>We propose SING, a novel framework that formulates the recovery of high-quality images from corrupted reconstructions as an inverse problem.
  arXiv  Detail & Related papers  (2025-03-16T12:32:11Z)
• Progressive Compression with Universally Quantized Diffusion Models [35.199627388957566]
  We explore the potential of diffusion models for progressive coding, resulting in a sequence of bits that can be incrementally transmitted and decoded.<n>Unlike prior work based on Gaussian diffusion or conditional diffusion models, we propose a new form of diffusion model with uniform noise in the forward process.<n>We obtain promising first results on image compression, achieving competitive rate-distortion and rate-realism results on a wide range of bit-rates with a single model.
  arXiv  Detail & Related papers  (2024-12-14T19:06:01Z)
• High Frequency Matters: Uncertainty Guided Image Compression with Wavelet Diffusion [4.76749587454871]
  We propose an efficient Uncertainty-Guided image compression approach with wavelet Diffusion (UGDiff)<n>Our approach focuses on high frequency compression via the wavelet transform, since high frequency components are crucial for reconstructing image details.<n> Comprehensive experiments on two benchmark datasets validate the effectiveness of UGDiff.
  arXiv  Detail & Related papers  (2024-07-17T13:21:31Z)
• Lossy Image Compression with Foundation Diffusion Models [10.407650300093923]
  In this work we formulate the removal of quantization error as a denoising task, using diffusion to recover lost information in the transmitted image latent. Our approach allows us to perform less than 10% of the full diffusion generative process and requires no architectural changes to the diffusion model.
  arXiv  Detail & Related papers  (2024-04-12T16:23:42Z)
• Correcting Diffusion-Based Perceptual Image Compression with Privileged End-to-End Decoder [49.01721042973929]
  This paper presents a diffusion-based image compression method that employs a privileged end-to-end decoder model as correction. Experiments demonstrate the superiority of our method in both distortion and perception compared with previous perceptual compression methods.
  arXiv  Detail & Related papers  (2024-04-07T10:57:54Z)
• Semantic Ensemble Loss and Latent Refinement for High-Fidelity Neural Image Compression [58.618625678054826]
  This study presents an enhanced neural compression method designed for optimal visual fidelity. We have trained our model with a sophisticated semantic ensemble loss, integrating Charbonnier loss, perceptual loss, style loss, and a non-binary adversarial loss. Our empirical findings demonstrate that this approach significantly improves the statistical fidelity of neural image compression.
  arXiv  Detail & Related papers  (2024-01-25T08:11:27Z)
• Transferable Learned Image Compression-Resistant Adversarial Perturbations [66.46470251521947]
  Adversarial attacks can readily disrupt the image classification system, revealing the vulnerability of DNN-based recognition tasks. We introduce a new pipeline that targets image classification models that utilize learned image compressors as pre-processing modules.
  arXiv  Detail & Related papers  (2024-01-06T03:03:28Z)
• Low-Light Image Enhancement with Wavelet-based Diffusion Models [50.632343822790006]
  Diffusion models have achieved promising results in image restoration tasks, yet suffer from time-consuming, excessive computational resource consumption, and unstable restoration. We propose a robust and efficient Diffusion-based Low-Light image enhancement approach, dubbed DiffLL.
  arXiv  Detail & Related papers  (2023-06-01T03:08:28Z)
• Lossy Image Compression with Conditional Diffusion Models [25.158390422252097]
  This paper outlines an end-to-end optimized lossy image compression framework using diffusion generative models. In contrast to VAE-based neural compression, where the (mean) decoder is a deterministic neural network, our decoder is a conditional diffusion model. Our approach yields stronger reported FID scores than the GAN-based model, while also yielding competitive performance with VAE-based models in several distortion metrics.
  arXiv  Detail & Related papers  (2022-09-14T21:53:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.