Efficient Evaluation of Quantization-Effects in Neural Codecs

• URL: http://arxiv.org/abs/2502.04770v1
• Date: Fri, 07 Feb 2025 09:11:19 GMT
• Title: Efficient Evaluation of Quantization-Effects in Neural Codecs
• Authors: Wolfgang Mack, Ahmed Mustafa, Rafał Łaganowski, Samer Hijazy,
• Abstract summary: Training neural codecs requires techniques to allow a non-zero gradient across the quantizer. This paper proposes an efficient evaluation framework for neural codecs using simulated data. We validate our findings against an internal neural audio gradient and against the state-of-the-art descript-audio-codec.
• Score: 4.897318643396687
• License:
• Abstract: Neural codecs, comprising an encoder, quantizer, and decoder, enable signal transmission at exceptionally low bitrates. Training these systems requires techniques like the straight-through estimator, soft-to-hard annealing, or statistical quantizer emulation to allow a non-zero gradient across the quantizer. Evaluating the effect of quantization in neural codecs, like the influence of gradient passing techniques on the whole system, is often costly and time-consuming due to training demands and the lack of affordable and reliable metrics. This paper proposes an efficient evaluation framework for neural codecs using simulated data with a defined number of bits and low-complexity neural encoders/decoders to emulate the non-linear behavior in larger networks. Our system is highly efficient in terms of training time and computational and hardware requirements, allowing us to uncover distinct behaviors in neural codecs. We propose a modification to stabilize training with the straight-through estimator based on our findings. We validate our findings against an internal neural audio codec and against the state-of-the-art descript-audio-codec.

Related papers

• Neuromorphic Auditory Perception by Neural Spiketrum [27.871072042280712]
  We introduce a neural spike coding model called spiketrumtemporal, to transform the time-varying analog signals into efficient spike patterns. The model provides a sparse and efficient coding scheme with precisely controllable spike rate that facilitates training of spiking neural networks in various auditory perception tasks.
  arXiv  Detail & Related papers  (2023-09-11T13:06:19Z)
• A Cryogenic Memristive Neural Decoder for Fault-tolerant Quantum Error Correction [0.0]
  We design and analyze a neural decoder based on an in-memory crossbar (IMC) architecture. We develop hardware-aware re-training methods to mitigate the fidelity loss. This work provides a pathway to scalable, fast, and low-power cryogenic IMC hardware for integrated fault-tolerant QEC.
  arXiv  Detail & Related papers  (2023-07-18T17:46:33Z)
• Neural network decoder for near-term surface-code experiments [0.7100520098029438]
  Neural-network decoders can achieve a lower logical error rate compared to conventional decoders. These decoders require no prior information about the physical error rates, making them highly adaptable.
  arXiv  Detail & Related papers  (2023-07-06T20:31:25Z)
• The END: An Equivariant Neural Decoder for Quantum Error Correction [73.4384623973809]
  We introduce a data efficient neural decoder that exploits the symmetries of the problem. We propose a novel equivariant architecture that achieves state of the art accuracy compared to previous neural decoders.
  arXiv  Detail & Related papers  (2023-04-14T19:46:39Z)
• Differentiable bit-rate estimation for neural-based video codec enhancement [2.592974861902384]
  Neural networks (NN) can improve standard video compression by pre- and post-processing the encoded video. For optimal NN training, the standard proxy needs to be replaced with a proxy that can provide derivatives of estimated bit-rate and distortion. This paper presents a new approach for bit-rate estimation that is similar to the type employed in training end-to-end neural codecs.
  arXiv  Detail & Related papers  (2023-01-24T01:36:07Z)
• Convolutional Neural Generative Coding: Scaling Predictive Coding to Natural Images [79.07468367923619]
  We develop convolutional neural generative coding (Conv-NGC) We implement a flexible neurobiologically-motivated algorithm that progressively refines latent state maps. We study the effectiveness of our brain-inspired neural system on the tasks of reconstruction and image denoising.
  arXiv  Detail & Related papers  (2022-11-22T06:42:41Z)
• High Fidelity Neural Audio Compression [92.4812002532009]
  We introduce a state-of-the-art real-time, high-fidelity, audio leveraging neural networks. It consists in a streaming encoder-decoder architecture with quantized latent space trained in an end-to-end fashion. We simplify and speed-up the training by using a single multiscale spectrogram adversary.
  arXiv  Detail & Related papers  (2022-10-24T17:52:02Z)
• NAF: Neural Attenuation Fields for Sparse-View CBCT Reconstruction [79.13750275141139]
  This paper proposes a novel and fast self-supervised solution for sparse-view CBCT reconstruction. The desired attenuation coefficients are represented as a continuous function of 3D spatial coordinates, parameterized by a fully-connected deep neural network. A learning-based encoder entailing hash coding is adopted to help the network capture high-frequency details.
  arXiv  Detail & Related papers  (2022-09-29T04:06:00Z)
• Achieving Low Complexity Neural Decoders via Iterative Pruning [33.774970857450086]
  We consider iterative pruning approaches to prune weights in neural decoders. Decoders with fewer number of weights can have lower latency and lower complexity. This will make neural decoders more suitable for mobile and other edge devices with limited computational power.
  arXiv  Detail & Related papers  (2021-12-11T18:33:08Z)
• SignalNet: A Low Resolution Sinusoid Decomposition and Estimation Network [79.04274563889548]
  We propose SignalNet, a neural network architecture that detects the number of sinusoids and estimates their parameters from quantized in-phase and quadrature samples. We introduce a worst-case learning threshold for comparing the results of our network relative to the underlying data distributions. In simulation, we find that our algorithm is always able to surpass the threshold for three-bit data but often cannot exceed the threshold for one-bit data.
  arXiv  Detail & Related papers  (2021-06-10T04:21:20Z)
• Ps and Qs: Quantization-aware pruning for efficient low latency neural network inference [56.24109486973292]
  We study the interplay between pruning and quantization during the training of neural networks for ultra low latency applications. We find that quantization-aware pruning yields more computationally efficient models than either pruning or quantization alone for our task.
  arXiv  Detail & Related papers  (2021-02-22T19:00:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.