L$_0$onie: Compressing COINs with L$_0$-constraints

• URL: http://arxiv.org/abs/2207.04144v1
• Date: Fri, 8 Jul 2022 22:24:56 GMT
• Title: L$_0$onie: Compressing COINs with L$_0$-constraints
• Authors: Juan Ramirez and Jose Gallego-Posada
• Abstract summary: Implicit Neural Representations (INR) have motivated research on domain-agnostic compression techniques. We propose a sparsity-constrained extension of the COIN compression method.
• Score: 0.4568777157687961
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Advances in Implicit Neural Representations (INR) have motivated research on domain-agnostic compression techniques. These methods train a neural network to approximate an object, and then store the weights of the trained model. For example, given an image, a network is trained to learn the mapping from pixel locations to RGB values. In this paper, we propose L$_0$onie, a sparsity-constrained extension of the COIN compression method. Sparsity allows to leverage the faster learning of overparameterized networks, while retaining the desirable compression rate of smaller models. Moreover, our constrained formulation ensures that the final model respects a pre-determined compression rate, dispensing of the need for expensive architecture search.

Related papers

• UniCompress: Enhancing Multi-Data Medical Image Compression with Knowledge Distillation [59.3877309501938]
  Implicit Neural Representation (INR) networks have shown remarkable versatility due to their flexible compression ratios. We introduce a codebook containing frequency domain information as a prior input to the INR network. This enhances the representational power of INR and provides distinctive conditioning for different image blocks.
  arXiv  Detail & Related papers  (2024-05-27T05:52:13Z)
• "Lossless" Compression of Deep Neural Networks: A High-dimensional Neural Tangent Kernel Approach [49.744093838327615]
  We provide a novel compression approach to wide and fully-connected emphdeep neural nets. Experiments on both synthetic and real-world data are conducted to support the advantages of the proposed compression scheme.
  arXiv  Detail & Related papers  (2024-03-01T03:46:28Z)
• Compression with Bayesian Implicit Neural Representations [16.593537431810237]
  We propose overfitting variational neural networks to the data and compressing an approximate posterior weight sample using relative entropy coding instead of quantizing and entropy coding it. Experiments show that our method achieves strong performance on image and audio compression while retaining simplicity.
  arXiv  Detail & Related papers  (2023-05-30T16:29:52Z)
• Rotation Invariant Quantization for Model Compression [7.633595230914364]
  Post-training Neural Network (NN) model compression is an attractive approach for deploying large, memory-consuming models on devices with limited memory resources. We suggest a Rotation-Invariant Quantization (RIQ) technique that utilizes a single parameter to quantize the entire NN model.
  arXiv  Detail & Related papers  (2023-03-03T10:53:30Z)
• Compressible-composable NeRF via Rank-residual Decomposition [21.92736190195887]
  Neural Radiance Field (NeRF) has emerged as a compelling method to represent 3D objects and scenes for photo-realistic rendering. We present a neural representation that enables efficient and convenient manipulation of models. Our method is able to achieve comparable rendering quality to state-of-the-art methods, while enabling extra capability of compression and composition.
  arXiv  Detail & Related papers  (2022-05-30T06:18:59Z)
• Estimating the Resize Parameter in End-to-end Learned Image Compression [50.20567320015102]
  We describe a search-free resizing framework that can further improve the rate-distortion tradeoff of recent learned image compression models. Our results show that our new resizing parameter estimation framework can provide Bjontegaard-Delta rate (BD-rate) improvement of about 10% against leading perceptual quality engines.
  arXiv  Detail & Related papers  (2022-04-26T01:35:02Z)
• Implicit Neural Representations for Image Compression [103.78615661013623]
  Implicit Neural Representations (INRs) have gained attention as a novel and effective representation for various data types. We propose the first comprehensive compression pipeline based on INRs including quantization, quantization-aware retraining and entropy coding. We find that our approach to source compression with INRs vastly outperforms similar prior work.
  arXiv  Detail & Related papers  (2021-12-08T13:02:53Z)
• Learning Scalable $\ell_\infty$-constrained Near-lossless Image Compression via Joint Lossy Image and Residual Compression [118.89112502350177]
  We propose a novel framework for learning $ell_infty$-constrained near-lossless image compression. We derive the probability model of the quantized residual by quantizing the learned probability model of the original residual.
  arXiv  Detail & Related papers  (2021-03-31T11:53:36Z)
• Training with Quantization Noise for Extreme Model Compression [57.51832088938618]
  We tackle the problem of producing compact models, maximizing their accuracy for a given model size. A standard solution is to train networks with Quantization Aware Training, where the weights are quantized during training and the gradients approximated with the Straight-Through Estimator. In this paper, we extend this approach to work beyond int8 fixed-point quantization with extreme compression methods.
  arXiv  Detail & Related papers  (2020-04-15T20:10:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.