From Algorithm to Hardware: A Survey on Efficient and Safe Deployment of Deep Neural Networks

• URL: http://arxiv.org/abs/2405.06038v1
• Date: Thu, 9 May 2024 18:17:25 GMT
• Title: From Algorithm to Hardware: A Survey on Efficient and Safe Deployment of Deep Neural Networks
• Authors: Xue Geng, Zhe Wang, Chunyun Chen, Qing Xu, Kaixin Xu, Chao Jin, Manas Gupta, Xulei Yang, Zhenghua Chen, Mohamed M. Sabry Aly, Jie Lin, Min Wu, Xiaoli Li,
• Abstract summary: Deep neural networks (DNNs) have been widely used in many artificial intelligence (AI) tasks. deploying them brings significant challenges due to the huge cost of memory, energy, and computation. Recently, there has been a surge in research of compression methods to achieve model efficiency while retaining the performance.
• Score: 23.928893359202753
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Deep neural networks (DNNs) have been widely used in many artificial intelligence (AI) tasks. However, deploying them brings significant challenges due to the huge cost of memory, energy, and computation. To address these challenges, researchers have developed various model compression techniques such as model quantization and model pruning. Recently, there has been a surge in research of compression methods to achieve model efficiency while retaining the performance. Furthermore, more and more works focus on customizing the DNN hardware accelerators to better leverage the model compression techniques. In addition to efficiency, preserving security and privacy is critical for deploying DNNs. However, the vast and diverse body of related works can be overwhelming. This inspires us to conduct a comprehensive survey on recent research toward the goal of high-performance, cost-efficient, and safe deployment of DNNs. Our survey first covers the mainstream model compression techniques such as model quantization, model pruning, knowledge distillation, and optimizations of non-linear operations. We then introduce recent advances in designing hardware accelerators that can adapt to efficient model compression approaches. Additionally, we discuss how homomorphic encryption can be integrated to secure DNN deployment. Finally, we discuss several issues, such as hardware evaluation, generalization, and integration of various compression approaches. Overall, we aim to provide a big picture of efficient DNNs, from algorithm to hardware accelerators and security perspectives.

Related papers

• Improving Spiking Neural Network Accuracy With Color Model Information Encoded Bit Planes [0.0]
  Spiking neural networks (SNNs) have emerged as a promising paradigm in computational neuroscience and artificial intelligence. We present a novel approach to enhance the performance of SNNs through a new encoding method.
  arXiv  Detail & Related papers  (2024-09-28T15:52:49Z)
• Model Agnostic Hybrid Sharding For Heterogeneous Distributed Inference [11.39873199479642]
  Nesa introduces a model-agnostic sharding framework designed for decentralized AI inference. Our framework uses blockchain-based deep neural network sharding to distribute computational tasks across a diverse network of nodes. Our results highlight the potential to democratize access to cutting-edge AI technologies.
  arXiv  Detail & Related papers  (2024-07-29T08:18:48Z)
• Inference Optimization of Foundation Models on AI Accelerators [68.24450520773688]
  Powerful foundation models, including large language models (LLMs), with Transformer architectures have ushered in a new era of Generative AI. As the number of model parameters reaches to hundreds of billions, their deployment incurs prohibitive inference costs and high latency in real-world scenarios. This tutorial offers a comprehensive discussion on complementary inference optimization techniques using AI accelerators.
  arXiv  Detail & Related papers  (2024-07-12T09:24:34Z)
• DNA Family: Boosting Weight-Sharing NAS with Block-Wise Supervisions [121.05720140641189]
  We develop a family of models with the distilling neural architecture (DNA) techniques. Our proposed DNA models can rate all architecture candidates, as opposed to previous works that can only access a sub- search space using algorithms. Our models achieve state-of-the-art top-1 accuracy of 78.9% and 83.6% on ImageNet for a mobile convolutional network and a small vision transformer, respectively.
  arXiv  Detail & Related papers  (2024-03-02T22:16:47Z)
• Design Automation for Fast, Lightweight, and Effective Deep Learning Models: A Survey [53.258091735278875]
  This survey covers studies of design automation techniques for deep learning models targeting edge computing. It offers an overview and comparison of key metrics that are used commonly to quantify the proficiency of models in terms of effectiveness, lightness, and computational costs. The survey proceeds to cover three categories of the state-of-the-art of deep model design automation techniques.
  arXiv  Detail & Related papers  (2022-08-22T12:12:43Z)
• DiverGet: A Search-Based Software Testing Approach for Deep Neural Network Quantization Assessment [10.18462284491991]
  Quantization is one of the most applied Deep Neural Network (DNN) compression strategies. We present DiverGet, a search-based testing framework for quantization assessment. We evaluate the performance of DiverGet on state-of-the-art DNNs applied to hyperspectral remote sensing images.
  arXiv  Detail & Related papers  (2022-07-13T15:27:51Z)
• DepthShrinker: A New Compression Paradigm Towards Boosting Real-Hardware Efficiency of Compact Neural Networks [29.46621102184345]
  We propose a framework dubbed DepthShrinker to develop hardware-friendly compact networks. Our framework delivers hardware-friendly compact networks that outperform both state-of-the-art efficient DNNs and compression techniques.
  arXiv  Detail & Related papers  (2022-06-02T02:32:47Z)
• FPGA-optimized Hardware acceleration for Spiking Neural Networks [69.49429223251178]
  This work presents the development of a hardware accelerator for an SNN, with off-line training, applied to an image recognition task. The design targets a Xilinx Artix-7 FPGA, using in total around the 40% of the available hardware resources. It reduces the classification time by three orders of magnitude, with a small 4.5% impact on the accuracy, if compared to its software, full precision counterpart.
  arXiv  Detail & Related papers  (2022-01-18T13:59:22Z)
• Quantized Neural Networks via {-1, +1} Encoding Decomposition and Acceleration [83.84684675841167]
  We propose a novel encoding scheme using -1, +1 to decompose quantized neural networks (QNNs) into multi-branch binary networks. We validate the effectiveness of our method on large-scale image classification, object detection, and semantic segmentation tasks.
  arXiv  Detail & Related papers  (2021-06-18T03:11:15Z)
• Adversarially Robust and Explainable Model Compression with On-Device Personalization for Text Classification [4.805959718658541]
  On-device Deep Neural Networks (DNNs) have recently gained more attention due to the increasing computing power of mobile devices and the number of applications in Computer Vision (CV) and Natural Language Processing (NLP) In NLP applications, although model compression has seen initial success, there are at least three major challenges yet to be addressed: adversarial robustness, explainability, and personalization. Here we attempt to tackle these challenges by designing a new training scheme for model compression and adversarial robustness, including the optimization of an explainable feature mapping objective. The resulting compressed model is personalized using on-device private training data via fine-
  arXiv  Detail & Related papers  (2021-01-10T15:06:55Z)
• Towards an Efficient and General Framework of Robust Training for Graph Neural Networks [96.93500886136532]
  Graph Neural Networks (GNNs) have made significant advances on several fundamental inference tasks. Despite GNNs' impressive performance, it has been observed that carefully crafted perturbations on graph structures lead them to make wrong predictions. We propose a general framework which leverages the greedy search algorithms and zeroth-order methods to obtain robust GNNs.
  arXiv  Detail & Related papers  (2020-02-25T15:17:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.