Latent-Shift: Gradient of Entropy Helps Neural Codecs

• URL: http://arxiv.org/abs/2308.00725v1
• Date: Tue, 1 Aug 2023 15:12:36 GMT
• Title: Latent-Shift: Gradient of Entropy Helps Neural Codecs
• Authors: Muhammet Balcilar, Bharath Bhushan Damodaran, Karam Naser, Franck Galpin and Pierre Hellier
• Abstract summary: We show that gradient of entropy is correlated with the gradient of the reconstruction error. We then demonstrate experimentally that this gradient can be used on various compression methods, leading to a $1-2%$ rate savings.
• Score: 4.449835214520727
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: End-to-end image/video codecs are getting competitive compared to traditional compression techniques that have been developed through decades of manual engineering efforts. These trainable codecs have many advantages over traditional techniques such as easy adaptation on perceptual distortion metrics and high performance on specific domains thanks to their learning ability. However, state of the art neural codecs does not take advantage of the existence of gradient of entropy in decoding device. In this paper, we theoretically show that gradient of entropy (available at decoder side) is correlated with the gradient of the reconstruction error (which is not available at decoder side). We then demonstrate experimentally that this gradient can be used on various compression methods, leading to a $1-2\%$ rate savings for the same quality. Our method is orthogonal to other improvements and brings independent rate savings.

Related papers

• Semantic Ensemble Loss and Latent Refinement for High-Fidelity Neural Image Compression [58.618625678054826]
  This study presents an enhanced neural compression method designed for optimal visual fidelity. We have trained our model with a sophisticated semantic ensemble loss, integrating Charbonnier loss, perceptual loss, style loss, and a non-binary adversarial loss. Our empirical findings demonstrate that this approach significantly improves the statistical fidelity of neural image compression.
  arXiv  Detail & Related papers  (2024-01-25T08:11:27Z)
• Accelerating Learnt Video Codecs with Gradient Decay and Layer-wise Distillation [17.980800481385195]
  We present a novel model-agnostic pruning scheme based on gradient decay and adaptive layer-wise distillation. Results confirm that our method yields up to 65% reduction in MACs and 2x speed-up with less than 0.3dB drop in BD-PSNR.
  arXiv  Detail & Related papers  (2023-12-05T09:26:09Z)
• DiffRate : Differentiable Compression Rate for Efficient Vision Transformers [98.33906104846386]
  Token compression aims to speed up large-scale vision transformers (e.g. ViTs) by pruning (dropping) or merging tokens. DiffRate is a novel token compression method that has several appealing properties prior arts do not have.
  arXiv  Detail & Related papers  (2023-05-29T10:15:19Z)
• Computationally-Efficient Neural Image Compression with Shallow Decoders [43.115831685920114]
  This paper takes a step forward towards closing the gap in decoding complexity by using a shallow or even linear decoding transform resembling that of JPEG. We exploit the often asymmetrical budget between encoding and decoding, by adopting more powerful encoder networks and iterative encoding.
  arXiv  Detail & Related papers  (2023-04-13T03:38:56Z)
• Video Coding Using Learned Latent GAN Compression [1.6058099298620423]
  We leverage the generative capacity of GANs such as StyleGAN to represent and compress a video. Each frame is inverted in the latent space of StyleGAN, from which the optimal compression is learned.
  arXiv  Detail & Related papers  (2022-07-09T19:07:43Z)
• Neural JPEG: End-to-End Image Compression Leveraging a Standard JPEG Encoder-Decoder [73.48927855855219]
  We propose a system that learns to improve the encoding performance by enhancing its internal neural representations on both the encoder and decoder ends. Experiments demonstrate that our approach successfully improves the rate-distortion performance over JPEG across various quality metrics.
  arXiv  Detail & Related papers  (2022-01-27T20:20:03Z)
• Learned transform compression with optimized entropy encoding [72.20409648915398]
  We consider the problem of learned transform compression where we learn both, the transform and the probability distribution over the discrete codes. We employ a soft relaxation of the quantization operation to allow for back-propagation of gradients and employ vector (rather than scalar) quantization of the latent codes.
  arXiv  Detail & Related papers  (2021-04-07T17:58:01Z)
• How to Exploit the Transferability of Learned Image Compression to Conventional Codecs [25.622863999901874]
  We show how learned image coding can be used as a surrogate to optimize an image for encoding. Our approach can remodel a conventional image to adjust for the MS-SSIM distortion with over 20% rate improvement without any decoding overhead.
  arXiv  Detail & Related papers  (2020-12-03T12:34:51Z)
• Conditional Entropy Coding for Efficient Video Compression [82.35389813794372]
  We propose a very simple and efficient video compression framework that only focuses on modeling the conditional entropy between frames. We first show that a simple architecture modeling the entropy between the image latent codes is as competitive as other neural video compression works and video codecs. We then propose a novel internal learning extension on top of this architecture that brings an additional 10% savings without trading off decoding speed.
  arXiv  Detail & Related papers  (2020-08-20T20:01:59Z)
• Content Adaptive and Error Propagation Aware Deep Video Compression [110.31693187153084]
  We propose a content adaptive and error propagation aware video compression system. Our method employs a joint training strategy by considering the compression performance of multiple consecutive frames instead of a single frame. Instead of using the hand-crafted coding modes in the traditional compression systems, we design an online encoder updating scheme in our system.
  arXiv  Detail & Related papers  (2020-03-25T09:04:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.