Constraint Guided Model Quantization of Neural Networks

• URL: http://arxiv.org/abs/2409.20138v1
• Date: Mon, 30 Sep 2024 09:41:16 GMT
• Title: Constraint Guided Model Quantization of Neural Networks
• Authors: Quinten Van Baelen, Peter Karsmakers,
• Abstract summary: Constraint Guided Model Quantization (CGMQ) is a quantization aware training algorithm that uses an upper bound on the computational resources and reduces the bit-widths of the parameters of the neural network. It is shown on MNIST that the performance of CGMQ is competitive with state-of-the-art quantization aware training algorithms.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Deploying neural networks on the edge has become increasingly important as deep learning is being applied in an increasing amount of applications. The devices on the edge are typically characterised as having small computational resources as large computational resources results in a higher energy consumption, which is impractical for these devices. To reduce the complexity of neural networks a wide range of quantization methods have been proposed in recent years. This work proposes Constraint Guided Model Quantization (CGMQ), which is a quantization aware training algorithm that uses an upper bound on the computational resources and reduces the bit-widths of the parameters of the neural network. CGMQ does not require the tuning of a hyperparameter to result in a mixed precision neural network that satisfies the predefined computational cost constraint, while prior work does. It is shown on MNIST that the performance of CGMQ is competitive with state-of-the-art quantization aware training algorithms, while guaranteeing the satisfaction of the cost constraint.

Related papers

• Generative AI-enabled Quantum Computing Networks and Intelligent Resource Allocation [80.78352800340032]
  Quantum computing networks execute large-scale generative AI computation tasks and advanced quantum algorithms. efficient resource allocation in quantum computing networks is a critical challenge due to qubit variability and network complexity. We introduce state-of-the-art reinforcement learning (RL) algorithms, from generative learning to quantum machine learning for optimal quantum resource allocation.
  arXiv  Detail & Related papers  (2024-01-13T17:16:38Z)
• Low Precision Quantization-aware Training in Spiking Neural Networks with Differentiable Quantization Function [0.5046831208137847]
  This work aims to bridge the gap between recent progress in quantized neural networks and spiking neural networks. It presents an extensive study on the performance of the quantization function, represented as a linear combination of sigmoid functions. The presented quantization function demonstrates the state-of-the-art performance on four popular benchmarks.
  arXiv  Detail & Related papers  (2023-05-30T09:42:05Z)
• CoNLoCNN: Exploiting Correlation and Non-Uniform Quantization for Energy-Efficient Low-precision Deep Convolutional Neural Networks [13.520972975766313]
  We propose a framework to enable energy-efficient low-precision deep convolutional neural network inference by exploiting non-uniform quantization of weights. We also propose a novel data representation format, Encoded Low-Precision Binary Signed Digit, to compress the bit-width of weights.
  arXiv  Detail & Related papers  (2022-07-31T01:34:56Z)
• Low-bit Shift Network for End-to-End Spoken Language Understanding [7.851607739211987]
  We propose the use of power-of-two quantization, which quantizes continuous parameters into low-bit power-of-two values. This reduces computational complexity by removing expensive multiplication operations and with the use of low-bit weights.
  arXiv  Detail & Related papers  (2022-07-15T14:34:22Z)
• CEG4N: Counter-Example Guided Neural Network Quantization Refinement [2.722899166098862]
  We propose Counter-Example Guided Neural Network Quantization Refinement (CEG4N) This technique combines search-based quantization and equivalence verification. We produce models with up to 72% better accuracy than state-of-the-art techniques.
  arXiv  Detail & Related papers  (2022-07-09T09:25:45Z)
• Optimizing Tensor Network Contraction Using Reinforcement Learning [86.05566365115729]
  We propose a Reinforcement Learning (RL) approach combined with Graph Neural Networks (GNN) to address the contraction ordering problem. The problem is extremely challenging due to the huge search space, the heavy-tailed reward distribution, and the challenging credit assignment. We show how a carefully implemented RL-agent that uses a GNN as the basic policy construct can address these challenges.
  arXiv  Detail & Related papers  (2022-04-18T21:45:13Z)
• Low-bit Quantization of Recurrent Neural Network Language Models Using Alternating Direction Methods of Multipliers [67.688697838109]
  This paper presents a novel method to train quantized RNNLMs from scratch using alternating direction methods of multipliers (ADMM) Experiments on two tasks suggest the proposed ADMM quantization achieved a model size compression factor of up to 31 times over the full precision baseline RNNLMs.
  arXiv  Detail & Related papers  (2021-11-29T09:30:06Z)
• A White Paper on Neural Network Quantization [20.542729144379223]
  We introduce state-of-the-art algorithms for mitigating the impact of quantization noise on the network's performance. We consider two main classes of algorithms: Post-Training Quantization (PTQ) and Quantization-Aware-Training (QAT)
  arXiv  Detail & Related papers  (2021-06-15T17:12:42Z)
• A Survey of Quantization Methods for Efficient Neural Network Inference [75.55159744950859]
  quantization is the problem of distributing continuous real-valued numbers over a fixed discrete set of numbers to minimize the number of bits required. It has come to the forefront in recent years due to the remarkable performance of Neural Network models in computer vision, natural language processing, and related areas. Moving from floating-point representations to low-precision fixed integer values represented in four bits or less holds the potential to reduce the memory footprint and latency by a factor of 16x.
  arXiv  Detail & Related papers  (2021-03-25T06:57:11Z)
• Ps and Qs: Quantization-aware pruning for efficient low latency neural network inference [56.24109486973292]
  We study the interplay between pruning and quantization during the training of neural networks for ultra low latency applications. We find that quantization-aware pruning yields more computationally efficient models than either pruning or quantization alone for our task.
  arXiv  Detail & Related papers  (2021-02-22T19:00:05Z)
• Widening and Squeezing: Towards Accurate and Efficient QNNs [125.172220129257]
  Quantization neural networks (QNNs) are very attractive to the industry because their extremely cheap calculation and storage overhead, but their performance is still worse than that of networks with full-precision parameters. Most of existing methods aim to enhance performance of QNNs especially binary neural networks by exploiting more effective training techniques. We address this problem by projecting features in original full-precision networks to high-dimensional quantization features.
  arXiv  Detail & Related papers  (2020-02-03T04:11:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.