Fixflow: A Framework to Evaluate Fixed-point Arithmetic in Light-Weight CNN Inference

• URL: http://arxiv.org/abs/2302.09564v1
• Date: Sun, 19 Feb 2023 12:48:52 GMT
• Title: Fixflow: A Framework to Evaluate Fixed-point Arithmetic in Light-Weight CNN Inference
• Authors: Farhad Taheri, Siavash Bayat-Sarmadi, Hatame Mosanaei-Boorani and Reza Taheri
• Abstract summary: Convolutional neural networks (CNN) are widely used in resource-constrained devices in IoT applications. This paper investigates the effect of different fixed-point hardware units on CNN inference accuracy.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Convolutional neural networks (CNN) are widely used in resource-constrained devices in IoT applications. In order to reduce the computational complexity and memory footprint, the resource-constrained devices use fixed-point representation. This representation consumes less area and energy in hardware with similar classification accuracy compared to the floating-point ones. However, to employ the low-precision fixed-point representation, various considerations to gain high accuracy are required. Although many quantization and re-training techniques are proposed to improve the inference accuracy, these approaches are time-consuming and require access to the entire dataset. This paper investigates the effect of different fixed-point hardware units on CNN inference accuracy. To this end, we provide a framework called Fixflow to evaluate the effect of fixed-point computations performed at hardware level on CNN classification accuracy. We can employ different fixed-point considerations at the hardware accelerators.This includes rounding methods and adjusting the precision of the fixed-point operation's result. Fixflow can determine the impact of employing different arithmetic units (such as truncated multipliers) on CNN classification accuracy. Moreover, we evaluate the energy and area consumption of these units in hardware accelerators. We perform experiments on two common MNIST and CIFAR-10 datasets. Our results show that employing different methods at the hardware level specially with low-precision, can significantly change the classification accuracy.

Related papers

• Low-Precision Floating-Point for Efficient On-Board Deep Neural Network Processing [0.9374652839580183]
  We study how to combine low precision (mini) floating-point arithmetic with a Quantization-Aware Training methodology. Our results show that 6-bit floating-point quantization for both weights and activations can compete with single-precision. An initial hardware study also confirms the potential impact of such low-precision floating-point designs.
  arXiv  Detail & Related papers  (2023-11-18T21:36:52Z)
• Cal-DETR: Calibrated Detection Transformer [67.75361289429013]
  We propose a mechanism for calibrated detection transformers (Cal-DETR), particularly for Deformable-DETR, UP-DETR and DINO. We develop an uncertainty-guided logit modulation mechanism that leverages the uncertainty to modulate the class logits. Results corroborate the effectiveness of Cal-DETR against the competing train-time methods in calibrating both in-domain and out-domain detections.
  arXiv  Detail & Related papers  (2023-11-06T22:13:10Z)
• Task-Oriented Over-the-Air Computation for Multi-Device Edge AI [57.50247872182593]
  6G networks for supporting edge AI features task-oriented techniques that focus on effective and efficient execution of AI task. Task-oriented over-the-air computation (AirComp) scheme is proposed in this paper for multi-device split-inference system.
  arXiv  Detail & Related papers  (2022-11-02T16:35:14Z)
• Green, Quantized Federated Learning over Wireless Networks: An Energy-Efficient Design [68.86220939532373]
  The finite precision level is captured through the use of quantized neural networks (QNNs) that quantize weights and activations in fixed-precision format. The proposed FL framework can reduce energy consumption until convergence by up to 70% compared to a baseline FL algorithm.
  arXiv  Detail & Related papers  (2022-07-19T16:37:24Z)
• AMED: Automatic Mixed-Precision Quantization for Edge Devices [3.5223695602582614]
  Quantized neural networks are well known for reducing the latency, power consumption, and model size without significant harm to the performance. Mixed-precision quantization offers better utilization of customized hardware that supports arithmetic operations at different bitwidths.
  arXiv  Detail & Related papers  (2022-05-30T21:23:22Z)
• FxP-QNet: A Post-Training Quantizer for the Design of Mixed Low-Precision DNNs with Dynamic Fixed-Point Representation [2.4149105714758545]
  We propose a novel framework referred to as the Fixed-Point Quantizer of deep neural Networks (FxP-QNet) FxP-QNet adapts the quantization level for each data-structure of each layer based on the trade-off between the network accuracy and the low-precision requirements. Results show that FxP-QNet-quantized AlexNet, VGG-16, and ResNet-18 reduce the overall memory requirements of their full-precision counterparts by 7.16x, 10.36x, and 6.44x with less than 0.95%, 0.95%, and 1.99%
  arXiv  Detail & Related papers  (2022-03-22T23:01:43Z)
• ZippyPoint: Fast Interest Point Detection, Description, and Matching through Mixed Precision Discretization [71.91942002659795]
  We investigate and adapt network quantization techniques to accelerate inference and enable its use on compute limited platforms. ZippyPoint, our efficient quantized network with binary descriptors, improves the network runtime speed, the descriptor matching speed, and the 3D model size. These improvements come at a minor performance degradation as evaluated on the tasks of homography estimation, visual localization, and map-free visual relocalization.
  arXiv  Detail & Related papers  (2022-03-07T18:59:03Z)
• On the Tradeoff between Energy, Precision, and Accuracy in Federated Quantized Neural Networks [68.52621234990728]
  Federated learning (FL) over wireless networks requires balancing between accuracy, energy efficiency, and precision. We propose a quantized FL framework that represents data with a finite level of precision in both local training and uplink transmission. Our framework can reduce energy consumption by up to 53% compared to a standard FL model.
  arXiv  Detail & Related papers  (2021-11-15T17:00:03Z)
• AQD: Towards Accurate Fully-Quantized Object Detection [94.06347866374927]
  We propose an Accurate Quantized object Detection solution, termed AQD, to get rid of floating-point computation. Our AQD achieves comparable or even better performance compared with the full-precision counterpart under extremely low-bit schemes.
  arXiv  Detail & Related papers  (2020-07-14T09:07:29Z)
• Accelerating Neural Network Inference by Overflow Aware Quantization [16.673051600608535]
  Inherited heavy computation of deep neural networks prevents their widespread applications. We propose an overflow aware quantization method by designing trainable adaptive fixed-point representation. With the proposed method, we are able to fully utilize the computing power to minimize the quantization loss and obtain optimized inference performance.
  arXiv  Detail & Related papers  (2020-05-27T11:56:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.