Exploiting Semantic and Pixel Representations for Ultra-Low Bitrate Image Compression

• URL: http://arxiv.org/abs/2606.01608v1
• Date: Mon, 01 Jun 2026 02:59:31 GMT
• Title: Exploiting Semantic and Pixel Representations for Ultra-Low Bitrate Image Compression
• Abstract summary: SPRDiff is a diffusion-based compression method that fully leverages both semantic and pixel representations.<n>We develop a triple-encoder architecture that utilizes high-fidelity features from the pretrained distortion-oriented and semantic-oriented encoders.<n>Experiments on benchmark datasets demonstrate that our method outperforms state-of-the-art approaches in the rate-distortion-perception tradeoff.
• Score: 46.54433210034761
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Most existing extreme compression methods fail to achieve an optimal rate-distortion-perception trade-off, as they typically prioritize perceptual fidelity and visual realism over pixel-level accuracy. Consequently, the resulting reconstructions often deviate noticeably from the originals. Ultra-low bitrate image compression is therefore crucial-not only for producing extremely compact representations but also for ensuring that reconstructed images remain semantically coherent and faithful to the source at the pixel level. To this end, we propose SPRDiff, a diffusion-based compression method that fully leverages both semantic and pixel representations, thereby enhancing reconstruction fidelity under ultra-low bitrate constraints. Specifically, we develop a triple-encoder architecture that utilizes high-fidelity features from the pretrained distortion-oriented and semantic-oriented encoders to compensate for the limited representations extracted by the frozen VAE encoder, thereby improving latent compression and entropy modeling. To further enhance the reconstruction fidelity of diffusion models, we introduce a distortion-aware reconstruction module with dual feature extraction. This module not only generates a coarse reconstruction that preserves the main structures, but also provides practical and accurate semantic- and pixel-level conditional signals to guide the diffusion model. Extensive experiments on benchmark datasets demonstrate that our method outperforms state-of-the-art approaches in the rate-distortion-perception tradeoff at extremely low bitrates (below 0.03 bpp), effectively preserving both perceptual quality and pixel-wise fidelity in the reconstructed images. We will release the source code and trained models at https://github.com/cshw2021/SPRDiff.

Related papers

• Faithful Extreme Image Rescaling with Learnable Reversible Transformation and Semantic Priors [46.54433210034761]
  FaithEIR is a diffusion-based framework for extreme image rescaling.<n>Inspired by singular value decomposition, we develop learnable reversible transformation.<n>To compensate for information loss due to quantization, we propose an adaptive detail prior.
  arXiv  Detail & Related papers  (2026-05-01T12:19:40Z)
• SODiff: Semantic-Oriented Diffusion Model for JPEG Compression Artifacts Removal [50.90827365790281]
  SODiff is a semantic-oriented one-step diffusion model for JPEG artifacts removal.<n>Our core idea is that effective restoration hinges on providing semantic-oriented guidance to the pre-trained diffusion model.<n>SAIPE extracts rich features from low-quality (LQ) images and projects them into an embedding space semantically aligned with that of the text encoder.
  arXiv  Detail & Related papers  (2025-08-10T13:48:07Z)
• Higher fidelity perceptual image and video compression with a latent conditioned residual denoising diffusion model [55.2480439325792]
  We propose a hybrid compression scheme optimized for perceptual quality, extending the approach of the CDC model with a decoder network.<n>We achieve up to +2dB PSNR fidelity improvements while maintaining comparable LPIPS and FID perceptual scores when compared with CDC.
  arXiv  Detail & Related papers  (2025-05-19T14:13:14Z)
• Ultra Lowrate Image Compression with Semantic Residual Coding and Compression-aware Diffusion [28.61304513668606]
  ResULIC is a residual-guided ultra lowrate image compression system.<n>It incorporates residual signals into both semantic retrieval and the diffusion-based generation process.<n>It achieves superior objective and subjective performance compared to state-of-the-art diffusion-based methods.
  arXiv  Detail & Related papers  (2025-05-13T06:51:23Z)
• Multi-Scale Invertible Neural Network for Wide-Range Variable-Rate Learned Image Compression [90.59962443790593]
  In this paper, we present a variable-rate image compression model based on invertible transform to overcome limitations.<n> Specifically, we design a lightweight multi-scale invertible neural network, which maps the input image into multi-scale latent representations.<n> Experimental results demonstrate that the proposed method achieves state-of-the-art performance compared to existing variable-rate methods.
  arXiv  Detail & Related papers  (2025-03-27T09:08:39Z)
• Leveraging Diffusion Knowledge for Generative Image Compression with Fractal Frequency-Aware Band Learning [16.077768397480902]
  generative image compression approaches produce detailed, realistic images instead of the only sharp-looking reconstructions.<n>We propose a novel deep learning-based generative image compression method injected with diffusion knowledge.<n>Our method achieves better distortions at high realism and better realism at low distortion than ever before.
  arXiv  Detail & Related papers  (2025-03-14T11:41:33Z)
• Controllable Distortion-Perception Tradeoff Through Latent Diffusion for Neural Image Compression [30.293252608423742]
  Neural image compression often faces a challenging trade-off among rate, distortion and perception.<n>We propose a novel approach that simultaneously addresses both aspects for a fixed neural image.<n>We can achieve more than 150% improvement in LPIPS-BDRate without sacrificing more than 1 dB in PSNR.
  arXiv  Detail & Related papers  (2024-12-16T02:09:32Z)
• Correcting Diffusion-Based Perceptual Image Compression with Privileged End-to-End Decoder [49.01721042973929]
  This paper presents a diffusion-based image compression method that employs a privileged end-to-end decoder model as correction. Experiments demonstrate the superiority of our method in both distortion and perception compared with previous perceptual compression methods.
  arXiv  Detail & Related papers  (2024-04-07T10:57:54Z)
• Semantic Ensemble Loss and Latent Refinement for High-Fidelity Neural Image Compression [58.618625678054826]
  This study presents an enhanced neural compression method designed for optimal visual fidelity. We have trained our model with a sophisticated semantic ensemble loss, integrating Charbonnier loss, perceptual loss, style loss, and a non-binary adversarial loss. Our empirical findings demonstrate that this approach significantly improves the statistical fidelity of neural image compression.
  arXiv  Detail & Related papers  (2024-01-25T08:11:27Z)
• Lossy Compression with Gaussian Diffusion [28.930398810600504]
  We describe a novel lossy compression approach called DiffC which is based on unconditional diffusion generative models. We implement a proof of concept and find that it works surprisingly well despite the lack of an encoder transform. We show that a flow-based reconstruction achieves a 3 dB gain over ancestral sampling at highs.
  arXiv  Detail & Related papers  (2022-06-17T16:46:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.