Controllable Distortion-Perception Tradeoff Through Latent Diffusion for Neural Image Compression

• URL: http://arxiv.org/abs/2412.11379v1
• Date: Mon, 16 Dec 2024 02:09:32 GMT
• Title: Controllable Distortion-Perception Tradeoff Through Latent Diffusion for Neural Image Compression
• Authors: Chuqin Zhou, Guo Lu, Jiangchuan Li, Xiangyu Chen, Zhengxue Cheng, Li Song, Wenjun Zhang,
• Abstract summary: Neural image compression often faces a challenging trade-off among rate, distortion and perception. We propose a novel approach that simultaneously addresses both aspects for a fixed neural image. We can achieve more than 150% improvement in LPIPS-BDRate without sacrificing more than 1 dB in PSNR.
• Score: 30.293252608423742
• License:
• Abstract: Neural image compression often faces a challenging trade-off among rate, distortion and perception. While most existing methods typically focus on either achieving high pixel-level fidelity or optimizing for perceptual metrics, we propose a novel approach that simultaneously addresses both aspects for a fixed neural image codec. Specifically, we introduce a plug-and-play module at the decoder side that leverages a latent diffusion process to transform the decoded features, enhancing either low distortion or high perceptual quality without altering the original image compression codec. Our approach facilitates fusion of original and transformed features without additional training, enabling users to flexibly adjust the balance between distortion and perception during inference. Extensive experimental results demonstrate that our method significantly enhances the pretrained codecs with a wide, adjustable distortion-perception range while maintaining their original compression capabilities. For instance, we can achieve more than 150% improvement in LPIPS-BDRate without sacrificing more than 1 dB in PSNR.

Related papers

• Rate-Distortion-Cognition Controllable Versatile Neural Image Compression [47.72668401825835]
  We propose a rate-distortion-cognition controllable versatile image compression method. Our method yields satisfactory ICM performance and flexible Rate-DistortionCognition controlling.
  arXiv  Detail & Related papers  (2024-07-16T13:17:51Z)
• PSC: Posterior Sampling-Based Compression [34.50287066865267]
  Posterior Sampling-based Compression (PSC) is a zero-shot compression method that leverages a pre-trained diffusion model as its sole neural network component. PSC constructs a transform that is adaptive to the image. We demonstrate that PSC's performance is comparable to established training-based methods in terms of rate, distortion, and perceptual quality.
  arXiv  Detail & Related papers  (2024-07-13T14:24:22Z)
• Once-for-All: Controllable Generative Image Compression with Dynamic Granularity Adaption [52.82508784748278]
  This paper proposes a Control Generative Image Compression framework, termed Control-GIC. Control-GIC is capable of fine-grained adaption across a broad spectrum while ensuring high-fidelity and generality compression. We develop a conditional decoder capable of retrieving historic multi-granularity representations according to encoded codes, and then reconstruct hierarchical features in the formalization of conditional probability.
  arXiv  Detail & Related papers  (2024-06-02T14:22:09Z)
• Enhancing Perception Quality in Remote Sensing Image Compression via Invertible Neural Network [10.427300958330816]
  Decoding remote sensing images to achieve high perceptual quality, particularly at lows, remains a significant challenge. We propose the invertible neural network-based remote sensing image compression (INN-RSIC) method. Our INN-RSIC significantly outperforms the existing state-of-the-art traditional and deep learning-based image compression methods in terms of perception quality.
  arXiv  Detail & Related papers  (2024-05-17T03:52:37Z)
• Correcting Diffusion-Based Perceptual Image Compression with Privileged End-to-End Decoder [49.01721042973929]
  This paper presents a diffusion-based image compression method that employs a privileged end-to-end decoder model as correction. Experiments demonstrate the superiority of our method in both distortion and perception compared with previous perceptual compression methods.
  arXiv  Detail & Related papers  (2024-04-07T10:57:54Z)
• Enhancing the Rate-Distortion-Perception Flexibility of Learned Image Codecs with Conditional Diffusion Decoders [7.485128109817576]
  We show that conditional diffusion models can lead to promising results in the generative compression task when used as a decoder. In this paper, we show that conditional diffusion models can lead to promising results in the generative compression task when used as a decoder.
  arXiv  Detail & Related papers  (2024-03-05T11:48:35Z)
• Semantic Ensemble Loss and Latent Refinement for High-Fidelity Neural Image Compression [58.618625678054826]
  This study presents an enhanced neural compression method designed for optimal visual fidelity. We have trained our model with a sophisticated semantic ensemble loss, integrating Charbonnier loss, perceptual loss, style loss, and a non-binary adversarial loss. Our empirical findings demonstrate that this approach significantly improves the statistical fidelity of neural image compression.
  arXiv  Detail & Related papers  (2024-01-25T08:11:27Z)
• Neural JPEG: End-to-End Image Compression Leveraging a Standard JPEG Encoder-Decoder [73.48927855855219]
  We propose a system that learns to improve the encoding performance by enhancing its internal neural representations on both the encoder and decoder ends. Experiments demonstrate that our approach successfully improves the rate-distortion performance over JPEG across various quality metrics.
  arXiv  Detail & Related papers  (2022-01-27T20:20:03Z)
• Substitutional Neural Image Compression [48.20906717052056]
  Substitutional Neural Image Compression (SNIC) is a general approach for enhancing any neural image compression model. It boosts compression performance toward a flexible distortion metric and enables bit-rate control using a single model instance.
  arXiv  Detail & Related papers  (2021-05-16T20:53:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.