GANDSE: Generative Adversarial Network based Design Space Exploration for Neural Network Accelerator Design

• URL: http://arxiv.org/abs/2208.00800v1
• Date: Mon, 1 Aug 2022 12:32:46 GMT
• Title: GANDSE: Generative Adversarial Network based Design Space Exploration for Neural Network Accelerator Design
• Authors: Lang Feng, Wenjian Liu, Chuliang Guo, Ke Tang, Cheng Zhuo, Zhongfeng Wang
• Abstract summary: We propose a neural network accelerator design automation framework named GANDSE. GANDSE is able to find the more optimized designs in negligible time compared with approaches including multilayer perceptron and deep reinforcement learning.
• Score: 27.290616313982323
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With the popularity of deep learning, the hardware implementation platform of deep learning has received increasing interest. Unlike the general purpose devices, e.g., CPU, or GPU, where the deep learning algorithms are executed at the software level, neural network hardware accelerators directly execute the algorithms to achieve higher both energy efficiency and performance improvements. However, as the deep learning algorithms evolve frequently, the engineering effort and cost of designing the hardware accelerators are greatly increased. To improve the design quality while saving the cost, design automation for neural network accelerators was proposed, where design space exploration algorithms are used to automatically search the optimized accelerator design within a design space. Nevertheless, the increasing complexity of the neural network accelerators brings the increasing dimensions to the design space. As a result, the previous design space exploration algorithms are no longer effective enough to find an optimized design. In this work, we propose a neural network accelerator design automation framework named GANDSE, where we rethink the problem of design space exploration, and propose a novel approach based on the generative adversarial network (GAN) to support an optimized exploration for high dimension large design space. The experiments show that GANDSE is able to find the more optimized designs in negligible time compared with approaches including multilayer perceptron and deep reinforcement learning.

Related papers

• AIRCHITECT v2: Learning the Hardware Accelerator Design Space through Unified Representations [3.6231171463908938]
  Design space exploration plays a crucial role in enabling custom hardware architectures. Recently, AIrchitect v1, the first attempt to address the limitations of DSE into a search-time classification problem.
  arXiv  Detail & Related papers  (2025-01-17T04:57:42Z)
• A Survey on Design Methodologies for Accelerating Deep Learning on Heterogeneous Architectures [9.215792818120903]
  This paper critically reviews influential tools and design methodologies for Deep Learning accelerators.<n>It covers hardware-software co-design, automated synthesis, domain-specific compilers, design space exploration, modeling, and simulation.
  arXiv  Detail & Related papers  (2023-11-29T17:10:16Z)
• Scaling Deep Networks with the Mesh Adaptive Direct Search algorithm [3.3073775218038883]
  We automate the design of a light deep neural network for image classification using the emphMesh Adaptive Direct Search(MADS) algorithm. Our tests show competitive compression rates with reduced numbers of trials.
  arXiv  Detail & Related papers  (2023-01-17T00:08:03Z)
• HARL: Hierarchical Adaptive Reinforcement Learning Based Auto Scheduler for Neural Networks [51.71682428015139]
  We propose HARL, a reinforcement learning-based auto-scheduler for efficient tensor program exploration. HarL improves the tensor operator performance by 22% and the search speed by 4.3x compared to the state-of-the-art auto-scheduler. Inference performance and search speed are also significantly improved on end-to-end neural networks.
  arXiv  Detail & Related papers  (2022-11-21T04:15:27Z)
• NASA: Neural Architecture Search and Acceleration for Hardware Inspired Hybrid Networks [24.95135135092478]
  We propose a Neural Architecture Search and Acceleration framework dubbed NASA. It enables automated multiplication-reduced DNN development and integrates a dedicated multiplication-reduced accelerator. Experiments consistently validate the advantages of NASA's algorithm-hardware co-design framework in terms of achievable accuracy and efficiency tradeoffs.
  arXiv  Detail & Related papers  (2022-10-24T16:03:42Z)
• A Semi-Decoupled Approach to Fast and Optimal Hardware-Software Co-Design of Neural Accelerators [22.69558355718029]
  Hardware-software co-design has been emerging to fully reap the benefits of flexible design spaces and optimize neural network performance. Such co-design enlarges the total search space to practically infinity and presents substantial challenges. We propose a emphsemi-decoupled approach to reduce the size of the total design space by orders of magnitude, yet without losing optimality.
  arXiv  Detail & Related papers  (2022-03-25T21:49:42Z)
• FPGA-optimized Hardware acceleration for Spiking Neural Networks [69.49429223251178]
  This work presents the development of a hardware accelerator for an SNN, with off-line training, applied to an image recognition task. The design targets a Xilinx Artix-7 FPGA, using in total around the 40% of the available hardware resources. It reduces the classification time by three orders of magnitude, with a small 4.5% impact on the accuracy, if compared to its software, full precision counterpart.
  arXiv  Detail & Related papers  (2022-01-18T13:59:22Z)
• AutoSpace: Neural Architecture Search with Less Human Interference [84.42680793945007]
  Current neural architecture search (NAS) algorithms still require expert knowledge and effort to design a search space for network construction. We propose a novel differentiable evolutionary framework named AutoSpace, which evolves the search space to an optimal one. With the learned search space, the performance of recent NAS algorithms can be improved significantly compared with using previously manually designed spaces.
  arXiv  Detail & Related papers  (2021-03-22T13:28:56Z)
• Evolving Search Space for Neural Architecture Search [70.71153433676024]
  We present a Neural Search-space Evolution (NSE) scheme that amplifies the results from the previous effort by maintaining an optimized search space subset. We achieve 77.3% top-1 retrain accuracy on ImageNet with 333M FLOPs, which yielded a state-of-the-art performance. When the latency constraint is adopted, our result also performs better than the previous best-performing mobile models with a 77.9% Top-1 retrain accuracy.
  arXiv  Detail & Related papers  (2020-11-22T01:11:19Z)
• Learned Hardware/Software Co-Design of Neural Accelerators [20.929918108940093]
  Deep learning software stacks and hardware accelerators are diverse and vast. Prior work considers software optimizations separately from hardware architectures, effectively reducing the search space. This paper casts the problem as hardware/software co-design, with the goal of automatically identifying desirable points in the joint design space.
  arXiv  Detail & Related papers  (2020-10-05T15:12:52Z)
• Generative Design by Reinforcement Learning: Enhancing the Diversity of Topology Optimization Designs [5.8010446129208155]
  This study proposes a reinforcement learning based generative design process, with reward functions maximizing the diversity of topology designs. We show that RL-based generative design produces a large number of diverse designs within a short inference time by exploiting GPU in a fully automated manner.
  arXiv  Detail & Related papers  (2020-08-17T06:50:47Z)
• Optimizing Memory Placement using Evolutionary Graph Reinforcement Learning [56.83172249278467]
  We introduce Evolutionary Graph Reinforcement Learning (EGRL), a method designed for large search spaces. We train and validate our approach directly on the Intel NNP-I chip for inference. We additionally achieve 28-78% speed-up compared to the native NNP-I compiler on all three workloads.
  arXiv  Detail & Related papers  (2020-07-14T18:50:12Z)
• DepthNet Nano: A Highly Compact Self-Normalizing Neural Network for Monocular Depth Estimation [76.90627702089357]
  DepthNet Nano is a compact deep neural network for monocular depth estimation designed using a human machine collaborative design strategy. The proposed DepthNet Nano possesses a highly efficient network architecture, while still achieving comparable performance with state-of-the-art networks.
  arXiv  Detail & Related papers  (2020-04-17T00:41:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.