Enhancing the Rate-Distortion-Perception Flexibility of Learned Image Codecs with Conditional Diffusion Decoders

• URL: http://arxiv.org/abs/2403.02887v1
• Date: Tue, 5 Mar 2024 11:48:35 GMT
• Title: Enhancing the Rate-Distortion-Perception Flexibility of Learned Image Codecs with Conditional Diffusion Decoders
• Authors: Daniele Mari, Simone Milani
• Abstract summary: We show that conditional diffusion models can lead to promising results in the generative compression task when used as a decoder. In this paper, we show that conditional diffusion models can lead to promising results in the generative compression task when used as a decoder.
• Score: 7.485128109817576
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Learned image compression codecs have recently achieved impressive compression performances surpassing the most efficient image coding architectures. However, most approaches are trained to minimize rate and distortion which often leads to unsatisfactory visual results at low bitrates since perceptual metrics are not taken into account. In this paper, we show that conditional diffusion models can lead to promising results in the generative compression task when used as a decoder, and that, given a compressed representation, they allow creating new tradeoff points between distortion and perception at the decoder side based on the sampling method.

Related papers

• Embedding Compression Distortion in Video Coding for Machines [67.97469042910855]
  Currently, video transmission serves not only the Human Visual System (HVS) for viewing but also machine perception for analysis. We propose a Compression Distortion Embedding (CDRE) framework, which extracts machine-perception-related distortion representation and embeds it into downstream models. Our framework can effectively boost the rate-task performance of existing codecs with minimal overhead in terms of execution time, and number of parameters.
  arXiv  Detail & Related papers  (2025-03-27T13:01:53Z)
• Multi-Scale Invertible Neural Network for Wide-Range Variable-Rate Learned Image Compression [90.59962443790593]
  In this paper, we present a variable-rate image compression model based on invertible transform to overcome limitations. Specifically, we design a lightweight multi-scale invertible neural network, which maps the input image into multi-scale latent representations. Experimental results demonstrate that the proposed method achieves state-of-the-art performance compared to existing variable-rate methods.
  arXiv  Detail & Related papers  (2025-03-27T09:08:39Z)
• Controllable Distortion-Perception Tradeoff Through Latent Diffusion for Neural Image Compression [30.293252608423742]
  Neural image compression often faces a challenging trade-off among rate, distortion and perception. We propose a novel approach that simultaneously addresses both aspects for a fixed neural image. We can achieve more than 150% improvement in LPIPS-BDRate without sacrificing more than 1 dB in PSNR.
  arXiv  Detail & Related papers  (2024-12-16T02:09:32Z)
• Once-for-All: Controllable Generative Image Compression with Dynamic Granularity Adaption [57.056311855630916]
  We propose a Controllable Generative Image Compression framework, Control-GIC. It is capable of fine-grained adaption across a broad spectrum while ensuring high-fidelity and generality compression. We develop a conditional conditionalization that can trace back to historic encoded multi-granularity representations.
  arXiv  Detail & Related papers  (2024-06-02T14:22:09Z)
• Correcting Diffusion-Based Perceptual Image Compression with Privileged End-to-End Decoder [49.01721042973929]
  This paper presents a diffusion-based image compression method that employs a privileged end-to-end decoder model as correction. Experiments demonstrate the superiority of our method in both distortion and perception compared with previous perceptual compression methods.
  arXiv  Detail & Related papers  (2024-04-07T10:57:54Z)
• Towards image compression with perfect realism at ultra-low bitrates [28.511327714128413]
  We dub our model PerCo for 'perceptual compression', and compare it to state-of-the-art codecs at rates from 0.1 down to 0.003 bits per pixel. We find that our model leads to reconstruction with state-of-the-art visual quality as measured by FID and KID.
  arXiv  Detail & Related papers  (2023-10-16T12:08:35Z)
• Extreme Image Compression using Fine-tuned VQGANs [43.43014096929809]
  We introduce vector quantization (VQ)-based generative models into the image compression domain. The codebook learned by the VQGAN model yields a strong expressive capacity. The proposed framework outperforms state-of-the-art codecs in terms of perceptual quality-oriented metrics.
  arXiv  Detail & Related papers  (2023-07-17T06:14:19Z)
• Lossy Compression with Gaussian Diffusion [28.930398810600504]
  We describe a novel lossy compression approach called DiffC which is based on unconditional diffusion generative models. We implement a proof of concept and find that it works surprisingly well despite the lack of an encoder transform. We show that a flow-based reconstruction achieves a 3 dB gain over ancestral sampling at highs.
  arXiv  Detail & Related papers  (2022-06-17T16:46:31Z)
• Estimating the Resize Parameter in End-to-end Learned Image Compression [50.20567320015102]
  We describe a search-free resizing framework that can further improve the rate-distortion tradeoff of recent learned image compression models. Our results show that our new resizing parameter estimation framework can provide Bjontegaard-Delta rate (BD-rate) improvement of about 10% against leading perceptual quality engines.
  arXiv  Detail & Related papers  (2022-04-26T01:35:02Z)
• Neural JPEG: End-to-End Image Compression Leveraging a Standard JPEG Encoder-Decoder [73.48927855855219]
  We propose a system that learns to improve the encoding performance by enhancing its internal neural representations on both the encoder and decoder ends. Experiments demonstrate that our approach successfully improves the rate-distortion performance over JPEG across various quality metrics.
  arXiv  Detail & Related papers  (2022-01-27T20:20:03Z)
• Modeling Lost Information in Lossy Image Compression [72.69327382643549]
  Lossy image compression is one of the most commonly used operators for digital images. We propose a novel invertible framework called Invertible Lossy Compression (ILC) to largely mitigate the information loss problem.
  arXiv  Detail & Related papers  (2020-06-22T04:04:56Z)
• Content Adaptive and Error Propagation Aware Deep Video Compression [110.31693187153084]
  We propose a content adaptive and error propagation aware video compression system. Our method employs a joint training strategy by considering the compression performance of multiple consecutive frames instead of a single frame. Instead of using the hand-crafted coding modes in the traditional compression systems, we design an online encoder updating scheme in our system.
  arXiv  Detail & Related papers  (2020-03-25T09:04:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.