DyCL: Dynamic Neural Network Compilation Via Program Rewriting and Graph Optimization

• URL: http://arxiv.org/abs/2307.04963v1
• Date: Tue, 11 Jul 2023 01:53:19 GMT
• Title: DyCL: Dynamic Neural Network Compilation Via Program Rewriting and Graph Optimization
• Authors: Simin Chen, Shiyi Wei, Cong Liu, Wei Yang
• Abstract summary: tool is a general approach that enables any existing DL compiler to successfully compile DyNNs. tool tackles the dynamic nature of DyNNs by converting a dynamic neural network into multiple sub-neural networks. compiled executables generated by tool exhibit significantly improved performance, running between $1.12times$ and $20.21times$ faster than the original DyNNs executed on general-purpose DL frameworks.
• Score: 8.701484095864744
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: DL compiler's primary function is to translate DNN programs written in high-level DL frameworks such as PyTorch and TensorFlow into portable executables. These executables can then be flexibly executed by the deployed host programs. However, existing DL compilers rely on a tracing mechanism, which involves feeding a runtime input to a neural network program and tracing the program execution paths to generate the computational graph necessary for compilation. Unfortunately, this mechanism falls short when dealing with modern dynamic neural networks (DyNNs) that possess varying computational graphs depending on the inputs. Consequently, conventional DL compilers struggle to accurately compile DyNNs into executable code. To address this limitation, we propose \tool, a general approach that enables any existing DL compiler to successfully compile DyNNs. \tool tackles the dynamic nature of DyNNs by introducing a compilation mechanism that redistributes the control and data flow of the original DNN programs during the compilation process. Specifically, \tool develops program analysis and program transformation techniques to convert a dynamic neural network into multiple sub-neural networks. Each sub-neural network is devoid of conditional statements and is compiled independently. Furthermore, \tool synthesizes a host module that models the control flow of the DyNNs and facilitates the invocation of the sub-neural networks. Our evaluation demonstrates the effectiveness of \tool, achieving a 100\% success rate in compiling all dynamic neural networks. Moreover, the compiled executables generated by \tool exhibit significantly improved performance, running between $1.12\times$ and $20.21\times$ faster than the original DyNNs executed on general-purpose DL frameworks.

Related papers

• Scaling Deep Learning Computation over the Inter-Core Connected Intelligence Processor with T10 [13.293273876476512]
  We present T10, the first DL compiler to exploit the inter-core communication bandwidth and distributed on-chip memory on AI chips. T10 makes globally optimized trade-offs between on-chip memory consumption and inter-core communication overhead. Our evaluation with a real inter-core connected AI chip, the Graphcore IPU, shows up to 3.3$times$ performance improvement.
  arXiv  Detail & Related papers  (2024-08-09T01:28:09Z)
• Boosting Neural Networks to Decompile Optimized Binaries [13.255618541522436]
  Decompilation aims to transform a low-level program language (LPL) into its functionally-equivalent high-level program language (HPL) We propose a novel learning-based approach named NeurDP, that targets compiler-optimized binaries.
  arXiv  Detail & Related papers  (2023-01-03T06:45:54Z)
• Intelligence Processing Units Accelerate Neuromorphic Learning [52.952192990802345]
  Spiking neural networks (SNNs) have achieved orders of magnitude improvement in terms of energy consumption and latency. We present an IPU-optimized release of our custom SNN Python package, snnTorch.
  arXiv  Detail & Related papers  (2022-11-19T15:44:08Z)
• Decompiling x86 Deep Neural Network Executables [20.91585339813852]
  BTD (Bin to DNN) is a decompiler for deep neural network (DNN) executables. We show that BTD can boost two representative attacks, adversarial example generation and knowledge stealing, against DNN executables.
  arXiv  Detail & Related papers  (2022-10-03T16:48:18Z)
• Dynamic Split Computing for Efficient Deep Edge Intelligence [78.4233915447056]
  We introduce dynamic split computing, where the optimal split location is dynamically selected based on the state of the communication channel. We show that dynamic split computing achieves faster inference in edge computing environments where the data rate and server load vary over time.
  arXiv  Detail & Related papers  (2022-05-23T12:35:18Z)
• An Adaptive Device-Edge Co-Inference Framework Based on Soft Actor-Critic [72.35307086274912]
  High-dimension parameter model and large-scale mathematical calculation restrict execution efficiency, especially for Internet of Things (IoT) devices. We propose a new Deep Reinforcement Learning (DRL)-Soft Actor Critic for discrete (SAC-d), which generates the emphexit point, emphexit point, and emphcompressing bits by soft policy iterations. Based on the latency and accuracy aware reward design, such an computation can well adapt to the complex environment like dynamic wireless channel and arbitrary processing, and is capable of supporting the 5G URL
  arXiv  Detail & Related papers  (2022-01-09T09:31:50Z)
• Learning to Execute Programs with Instruction Pointer Attention Graph Neural Networks [55.98291376393561]
  Graph neural networks (GNNs) have emerged as a powerful tool for learning software engineering tasks. Recurrent neural networks (RNNs) are well-suited to long sequential chains of reasoning, but they do not naturally incorporate program structure. We introduce a novel GNN architecture, the Instruction Pointer Attention Graph Neural Networks (IPA-GNN), which improves systematic generalization on the task of learning to execute programs.
  arXiv  Detail & Related papers  (2020-10-23T19:12:30Z)
• PolyDL: Polyhedral Optimizations for Creation of High Performance DL primitives [55.79741270235602]
  We present compiler algorithms to automatically generate high performance implementations of Deep Learning primitives. We develop novel data reuse analysis algorithms using the polyhedral model. We also show that such a hybrid compiler plus a minimal library-use approach results in state-of-the-art performance.
  arXiv  Detail & Related papers  (2020-06-02T06:44:09Z)
• PatDNN: Achieving Real-Time DNN Execution on Mobile Devices with Pattern-based Weight Pruning [57.20262984116752]
  We introduce a new dimension, fine-grained pruning patterns inside the coarse-grained structures, revealing a previously unknown point in design space. With the higher accuracy enabled by fine-grained pruning patterns, the unique insight is to use the compiler to re-gain and guarantee high hardware efficiency.
  arXiv  Detail & Related papers  (2020-01-01T04:52:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.