2-in-1 Accelerator: Enabling Random Precision Switch for Winning Both Adversarial Robustness and Efficiency

• URL: http://arxiv.org/abs/2109.05223v1
• Date: Sat, 11 Sep 2021 08:51:01 GMT
• Title: 2-in-1 Accelerator: Enabling Random Precision Switch for Winning Both Adversarial Robustness and Efficiency
• Authors: Yonggan Fu, Yang Zhao, Qixuan Yu, Chaojian Li, Yingyan Lin
• Abstract summary: We propose a 2-in-1 Accelerator aiming at winning both the adversarial robustness and efficiency of DNN accelerators. Specifically, we first propose a Random Precision Switch (RPS) algorithm that can effectively defend DNNs against adversarial attacks. Furthermore, we propose a new precision-scalable accelerator featuring (1) a new precision-scalable unit architecture.
• Score: 26.920864182619844
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The recent breakthroughs of deep neural networks (DNNs) and the advent of billions of Internet of Things (IoT) devices have excited an explosive demand for intelligent IoT devices equipped with domain-specific DNN accelerators. However, the deployment of DNN accelerator enabled intelligent functionality into real-world IoT devices still remains particularly challenging. First, powerful DNNs often come at prohibitive complexities, whereas IoT devices often suffer from stringent resource constraints. Second, while DNNs are vulnerable to adversarial attacks especially on IoT devices exposed to complex real-world environments, many IoT applications require strict security. Existing DNN accelerators mostly tackle only one of the two aforementioned challenges (i.e., efficiency or adversarial robustness) while neglecting or even sacrificing the other. To this end, we propose a 2-in-1 Accelerator, an integrated algorithm-accelerator co-design framework aiming at winning both the adversarial robustness and efficiency of DNN accelerators. Specifically, we first propose a Random Precision Switch (RPS) algorithm that can effectively defend DNNs against adversarial attacks by enabling random DNN quantization as an in-situ model switch. Furthermore, we propose a new precision-scalable accelerator featuring (1) a new precision-scalable MAC unit architecture which spatially tiles the temporal MAC units to boost both the achievable efficiency and flexibility and (2) a systematically optimized dataflow that is searched by our generic accelerator optimizer. Extensive experiments and ablation studies validate that our 2-in-1 Accelerator can not only aggressively boost both the adversarial robustness and efficiency of DNN accelerators under various attacks, but also naturally support instantaneous robustness-efficiency trade-offs adapting to varied resources without the necessity of DNN retraining.

Related papers

• DNN Partitioning, Task Offloading, and Resource Allocation in Dynamic Vehicular Networks: A Lyapunov-Guided Diffusion-Based Reinforcement Learning Approach [49.56404236394601]
  We formulate the problem of joint DNN partitioning, task offloading, and resource allocation in Vehicular Edge Computing. Our objective is to minimize the DNN-based task completion time while guaranteeing the system stability over time. We propose a Multi-Agent Diffusion-based Deep Reinforcement Learning (MAD2RL) algorithm, incorporating the innovative use of diffusion models.
  arXiv  Detail & Related papers  (2024-06-11T06:31:03Z)
• FireFly v2: Advancing Hardware Support for High-Performance Spiking Neural Network with a Spatiotemporal FPGA Accelerator [8.0611988136866]
  Spiking Neural Networks (SNNs) are expected to be a promising alternative to Artificial Neural Networks (ANNs) Specialized SNN hardware offers clear advantages over general-purpose devices in terms of power and performance. FireFly v2, an FPGA SNN accelerator, can address the issue of non-spike operation in current SOTA SNN algorithms.
  arXiv  Detail & Related papers  (2023-09-28T04:17:02Z)
• FPGA-optimized Hardware acceleration for Spiking Neural Networks [69.49429223251178]
  This work presents the development of a hardware accelerator for an SNN, with off-line training, applied to an image recognition task. The design targets a Xilinx Artix-7 FPGA, using in total around the 40% of the available hardware resources. It reduces the classification time by three orders of magnitude, with a small 4.5% impact on the accuracy, if compared to its software, full precision counterpart.
  arXiv  Detail & Related papers  (2022-01-18T13:59:22Z)
• O-HAS: Optical Hardware Accelerator Search for Boosting Both Acceleration Performance and Development Speed [13.41883640945134]
  O-HAS consists of two integrated enablers: (1) an O-Cost Predictor, which can accurately yet efficiently predict an optical accelerator's energy and latency based on the DNN model parameters and the optical accelerator design; and (2) an O-Search Engine, which can automatically explore the large design space of optical DNN accelerators. Experiments and ablation studies consistently validate the effectiveness of both our O-Cost Predictor and O-Search Engine.
  arXiv  Detail & Related papers  (2021-08-17T09:50:14Z)
• InstantNet: Automated Generation and Deployment of Instantaneously Switchable-Precision Networks [65.78061366594106]
  We propose InstantNet to automatically generate and deploy instantaneously switchable-precision networks which operate at variable bit-widths. In experiments, the proposed InstantNet consistently outperforms state-of-the-art designs.
  arXiv  Detail & Related papers  (2021-04-22T04:07:43Z)
• Random and Adversarial Bit Error Robustness: Energy-Efficient and Secure DNN Accelerators [105.60654479548356]
  We show that a combination of robust fixed-point quantization, weight clipping, as well as random bit error training (RandBET) improves robustness against random or adversarial bit errors in quantized DNN weights significantly. This leads to high energy savings for low-voltage operation as well as low-precision quantization, but also improves security of DNN accelerators.
  arXiv  Detail & Related papers  (2021-04-16T19:11:14Z)
• DNA: Differentiable Network-Accelerator Co-Search [36.68587348474986]
  We propose DNA, a Differentiable Network-Accelerator co-search framework for automatically searching for matched networks and accelerators. Specifically, DNA integrates two enablers: (1) a generic design space for DNN accelerators and compatible with DNN frameworks such as PyTorch to enable algorithmic exploration. Experiments and ablation studies show that the matched networks and accelerators generated by DNA consistently outperform state-of-the-art (SOTA) DNNs and accelerators.
  arXiv  Detail & Related papers  (2020-10-28T05:57:16Z)
• Sparsity Turns Adversarial: Energy and Latency Attacks on Deep Neural Networks [3.9193443389004887]
  Adrial attacks have exposed serious vulnerabilities in Deep Neural Networks (DNNs) We propose and demonstrate sparsity attacks, which adversarial modify a DNN's inputs so as to reduce sparsity in its internal activation values. We launch both white-box and black-box versions of adversarial sparsity attacks and demonstrate that they decrease activation sparsity by up to 1.82x.
  arXiv  Detail & Related papers  (2020-06-14T21:02:55Z)
• SmartExchange: Trading Higher-cost Memory Storage/Access for Lower-cost Computation [97.78417228445883]
  We present SmartExchange, an algorithm- hardware co-design framework for energy-efficient inference of deep neural networks (DNNs) We develop a novel algorithm to enforce a specially favorable DNN weight structure, where each layerwise weight matrix can be stored as the product of a small basis matrix and a large sparse coefficient matrix whose non-zero elements are all power-of-2. We further design a dedicated accelerator to fully utilize the SmartExchange-enforced weights to improve both energy efficiency and latency performance.
  arXiv  Detail & Related papers  (2020-05-07T12:12:49Z)
• DNN-Chip Predictor: An Analytical Performance Predictor for DNN Accelerators with Various Dataflows and Hardware Architectures [30.689015188050405]
  The recent breakthroughs in deep neural networks (DNNs) have spurred a tremendously increased demand for DNN accelerators. DNN-Chip Predictor is an analytical performance predictor which can accurately predict DNN accelerators' energy, throughput, and latency prior to their actual implementation.
  arXiv  Detail & Related papers  (2020-02-26T02:59:18Z)
• PatDNN: Achieving Real-Time DNN Execution on Mobile Devices with Pattern-based Weight Pruning [57.20262984116752]
  We introduce a new dimension, fine-grained pruning patterns inside the coarse-grained structures, revealing a previously unknown point in design space. With the higher accuracy enabled by fine-grained pruning patterns, the unique insight is to use the compiler to re-gain and guarantee high hardware efficiency.
  arXiv  Detail & Related papers  (2020-01-01T04:52:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.