Differentiable Vector Quantization for Rate-Distortion Optimization of Generative Image Compression

• URL: http://arxiv.org/abs/2604.10546v1
• Date: Sun, 12 Apr 2026 09:25:24 GMT
• Title: Differentiable Vector Quantization for Rate-Distortion Optimization of Generative Image Compression
• Abstract summary: We propose RDVQ, a unified framework that enables end-to-end RD optimization for low-bitrate image compression.<n>We develop an autoregressive entropy model that supports accurate entropy modeling and test-time rate control.<n>Compared with RD2K-val, RDVQ reduces by up to 75.71% on DISTS and 37.63% on LPIPS on DIV2K-val.
• Score: 53.041405703471895
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The rapid growth of visual data under stringent storage and bandwidth constraints makes extremely low-bitrate image compression increasingly important. While Vector Quantization (VQ) offers strong structural fidelity, existing methods lack a principled mechanism for joint rate-distortion (RD) optimization due to the disconnect between representation learning and entropy modeling. We propose RDVQ, a unified framework that enables end-to-end RD optimization for VQ-based compression via a differentiable relaxation of the codebook distribution, allowing the entropy loss to directly shape the latent prior. We further develop an autoregressive entropy model that supports accurate entropy modeling and test-time rate control. Extensive experiments demonstrate that RDVQ achieves strong performance at extremely low bitrates with a lightweight architecture, attaining competitive or superior perceptual quality with significantly fewer parameters. Compared with RDEIC, RDVQ reduces bitrate by up to 75.71% on DISTS and 37.63% on LPIPS on DIV2K-val. Beyond empirical gains, RDVQ introduces an entropy-constrained formulation of VQ, highlighting the potential for a more unified view of image tokenization and compression. The code will be available at https://github.com/CVL-UESTC/RDVQ.

Related papers

• Efficient Learned Image Compression without Entropy Coding [62.01837001379442]
  We present Entropy-Coding Free Learned Image Compression (EF-LIC)<n>EF-LIC is a multi-rate framework that generates compact representation by removing statistical and correlation redundancy with low coding latency.<n>Experiments show EF-LIC achieves up to 67.86% reduction over MS-ILLM on Kodak with LPIPS.
  arXiv  Detail & Related papers  (2026-05-22T07:37:30Z)
• ARCHE: Autoregressive Residual Compression with Hyperprior and Excitation [1.376408511310322]
  ARCHE is an end-to-end learned image compression framework that balances modeling accuracy and computational efficiency.<n>It attains state-of-the-art rate-distortion efficiency without relying on recurrent or transformer-based components.<n>Visual comparisons confirm sharper textures and improved color fidelity, particularly at lower bit rates.
  arXiv  Detail & Related papers  (2026-03-10T19:32:10Z)
• HVQ-CGIC: Enabling Hyperprior Entropy Modeling for VQ-Based Controllable Generative Image Compression [0.0]
  We introduce a Controllable Generative Image Compression framework based on a VQ Hyperprior.<n>HVQ-CGIC rigorously derives the mathematical foundation for introducing a hyperprior to the VQ indices entropy model.<n>It achieves a significant advantage in rate-distortion (RD) performance compared to current state-of-the-art (SOTA) generative compression methods.
  arXiv  Detail & Related papers  (2025-12-08T06:10:07Z)
• Rethinking Autoregressive Models for Lossless Image Compression via Hierarchical Parallelism and Progressive Adaptation [75.58269386927076]
  Autoregressive (AR) models are often dismissed as impractical due to prohibitive computational cost.<n>This work re-thinks this paradigm, introducing a framework built on hierarchical parallelism and progressive adaptation.<n> Experiments on diverse datasets (natural, satellite, medical) validate that our method achieves new state-of-the-art compression.
  arXiv  Detail & Related papers  (2025-11-14T06:27:58Z)
• Ultra Lowrate Image Compression with Semantic Residual Coding and Compression-aware Diffusion [28.61304513668606]
  ResULIC is a residual-guided ultra lowrate image compression system.<n>It incorporates residual signals into both semantic retrieval and the diffusion-based generation process.<n>It achieves superior objective and subjective performance compared to state-of-the-art diffusion-based methods.
  arXiv  Detail & Related papers  (2025-05-13T06:51:23Z)
• Multi-Scale Invertible Neural Network for Wide-Range Variable-Rate Learned Image Compression [90.59962443790593]
  In this paper, we present a variable-rate image compression model based on invertible transform to overcome limitations.<n> Specifically, we design a lightweight multi-scale invertible neural network, which maps the input image into multi-scale latent representations.<n> Experimental results demonstrate that the proposed method achieves state-of-the-art performance compared to existing variable-rate methods.
  arXiv  Detail & Related papers  (2025-03-27T09:08:39Z)
• VRVVC: Variable-Rate NeRF-Based Volumetric Video Compression [59.14355576912495]
  NeRF-based video has revolutionized visual media by delivering photorealistic Free-Viewpoint Video (FVV) experiences.<n>The substantial data volumes pose significant challenges for storage and transmission.<n>We propose VRVVC, a novel end-to-end joint variable-rate framework for video compression.
  arXiv  Detail & Related papers  (2024-12-16T01:28:04Z)
• Once-for-All: Controllable Generative Image Compression with Dynamic Granularity Adaptation [52.82508784748278]
  This paper proposes a Control Generative Image Compression framework, termed Control-GIC.<n>Control-GIC is capable of fine-grained adaption across a broad spectrum while ensuring high-fidelity and generality compression.<n>Our experiments show that Control-GIC allows highly flexible and controllable adaption where the results demonstrate its superior performance over recent state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-06-02T14:22:09Z)
• Extreme Image Compression using Fine-tuned VQGANs [43.43014096929809]
  We introduce vector quantization (VQ)-based generative models into the image compression domain. The codebook learned by the VQGAN model yields a strong expressive capacity. The proposed framework outperforms state-of-the-art codecs in terms of perceptual quality-oriented metrics.
  arXiv  Detail & Related papers  (2023-07-17T06:14:19Z)
• LVQAC: Lattice Vector Quantization Coupled with Spatially Adaptive Companding for Efficient Learned Image Compression [24.812267280543693]
  We present a novel Lattice Vector Quantization scheme coupled with a spatially Adaptive Companding (LVQAC) mapping. For any end-to-end CNN image compression models, replacing uniform quantizer by LVQAC achieves better rate-distortion performance without significantly increasing the model complexity.
  arXiv  Detail & Related papers  (2023-03-25T23:34:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.