Exploiting Latent Properties to Optimize Neural Codecs

• URL: http://arxiv.org/abs/2501.01231v1
• Date: Thu, 02 Jan 2025 12:45:31 GMT
• Title: Exploiting Latent Properties to Optimize Neural Codecs
• Authors: Muhammet Balcilar, Bharath Bhushan Damodaran, Karam Naser, Franck Galpin, Pierre Hellier,
• Abstract summary: We propose to leverage the benefits of vector quantization and the entropy gradient to improve the performance of off-the-shelf codecs. We show that these approaches save between 1 to 3% of the rate for the same quality across various pretrained methods.
• Score: 5.725546754007691
• License:
• Abstract: End-to-end image and video codecs are becoming increasingly competitive, compared to traditional compression techniques that have been developed through decades of manual engineering efforts. These trainable codecs have many advantages over traditional techniques, such as their straightforward adaptation to perceptual distortion metrics and high performance in specific fields thanks to their learning ability. However, current state-of-the-art neural codecs do not fully exploit the benefits of vector quantization and the existence of the entropy gradient in decoding devices. In this paper, we propose to leverage these two properties (vector quantization and entropy gradient) to improve the performance of off-the-shelf codecs. Firstly, we demonstrate that using non-uniform scalar quantization cannot improve performance over uniform quantization. We thus suggest using predefined optimal uniform vector quantization to improve performance. Secondly, we show that the entropy gradient, available at the decoder, is correlated with the reconstruction error gradient, which is not available at the decoder. We therefore use the former as a proxy to enhance compression performance. Our experimental results show that these approaches save between 1 to 3% of the rate for the same quality across various pretrained methods. In addition, the entropy gradient based solution improves traditional codec performance significantly as well.

Related papers

• VRVQ: Variable Bitrate Residual Vector Quantization for Audio Compression [29.368893236587343]
  Recent neural audio compression models have progressively adopted residual vector quantization (RVQ) These models employ a fixed number of codebooks per frame, which can be suboptimal in terms of rate-distortion tradeoffs. We propose variable RVQ (VRVQ) for audio codecs, which allows for more efficient coding by adapting the number of codebooks used per frame.
  arXiv  Detail & Related papers  (2024-10-08T13:18:24Z)
• Prediction and Reference Quality Adaptation for Learned Video Compression [54.58691829087094]
  Temporal prediction is one of the most important technologies for video compression. Traditional video codecs adaptively decide the optimal coding mode according to the prediction quality and reference quality. We propose a confidence-based prediction quality adaptation (PQA) module and a reference quality adaptation (RQA) module.
  arXiv  Detail & Related papers  (2024-06-20T09:03:26Z)
• ContextGS: Compact 3D Gaussian Splatting with Anchor Level Context Model [77.71796503321632]
  We introduce a context model in the anchor level for 3DGS representation, yielding an impressive size reduction of over 100 times compared to vanilla 3DGS. Our work pioneers the context model in the anchor level for 3DGS representation, yielding an impressive size reduction of over 100 times compared to vanilla 3DGS and 15 times compared to the most recent state-of-the-art work Scaffold-GS.
  arXiv  Detail & Related papers  (2024-05-31T09:23:39Z)
• Accelerating Learnt Video Codecs with Gradient Decay and Layer-wise Distillation [17.980800481385195]
  We present a novel model-agnostic pruning scheme based on gradient decay and adaptive layer-wise distillation. Results confirm that our method yields up to 65% reduction in MACs and 2x speed-up with less than 0.3dB drop in BD-PSNR.
  arXiv  Detail & Related papers  (2023-12-05T09:26:09Z)
• Latent-Shift: Gradient of Entropy Helps Neural Codecs [4.449835214520727]
  We show that gradient of entropy is correlated with the gradient of the reconstruction error. We then demonstrate experimentally that this gradient can be used on various compression methods, leading to a $1-2%$ rate savings.
  arXiv  Detail & Related papers  (2023-08-01T15:12:36Z)
• Quick Dense Retrievers Consume KALE: Post Training Kullback Leibler Alignment of Embeddings for Asymmetrical dual encoders [89.29256833403169]
  We introduce Kullback Leibler Alignment of Embeddings (KALE), an efficient and accurate method for increasing the inference efficiency of dense retrieval methods. KALE extends traditional Knowledge Distillation after bi-encoder training, allowing for effective query encoder compression without full retraining or index generation. Using KALE and asymmetric training, we can generate models which exceed the performance of DistilBERT despite having 3x faster inference.
  arXiv  Detail & Related papers  (2023-03-31T15:44:13Z)
• Differentiable bit-rate estimation for neural-based video codec enhancement [2.592974861902384]
  Neural networks (NN) can improve standard video compression by pre- and post-processing the encoded video. For optimal NN training, the standard proxy needs to be replaced with a proxy that can provide derivatives of estimated bit-rate and distortion. This paper presents a new approach for bit-rate estimation that is similar to the type employed in training end-to-end neural codecs.
  arXiv  Detail & Related papers  (2023-01-24T01:36:07Z)
• Neural JPEG: End-to-End Image Compression Leveraging a Standard JPEG Encoder-Decoder [73.48927855855219]
  We propose a system that learns to improve the encoding performance by enhancing its internal neural representations on both the encoder and decoder ends. Experiments demonstrate that our approach successfully improves the rate-distortion performance over JPEG across various quality metrics.
  arXiv  Detail & Related papers  (2022-01-27T20:20:03Z)
• Learned transform compression with optimized entropy encoding [72.20409648915398]
  We consider the problem of learned transform compression where we learn both, the transform and the probability distribution over the discrete codes. We employ a soft relaxation of the quantization operation to allow for back-propagation of gradients and employ vector (rather than scalar) quantization of the latent codes.
  arXiv  Detail & Related papers  (2021-04-07T17:58:01Z)
• Content Adaptive and Error Propagation Aware Deep Video Compression [110.31693187153084]
  We propose a content adaptive and error propagation aware video compression system. Our method employs a joint training strategy by considering the compression performance of multiple consecutive frames instead of a single frame. Instead of using the hand-crafted coding modes in the traditional compression systems, we design an online encoder updating scheme in our system.
  arXiv  Detail & Related papers  (2020-03-25T09:04:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.