Cross-Modality Program Representation Learning for Electronic Design Automation with High-Level Synthesis

• URL: http://arxiv.org/abs/2406.09606v3
• Date: Wed, 17 Jul 2024 22:08:51 GMT
• Title: Cross-Modality Program Representation Learning for Electronic Design Automation with High-Level Synthesis
• Authors: Zongyue Qin, Yunsheng Bai, Atefeh Sohrabizadeh, Zijian Ding, Ziniu Hu, Yizhou Sun, Jason Cong,
• Abstract summary: Domain-specific accelerators (DSAs) have gained popularity for applications such as deep learning and autonomous driving. We propose ProgSG, a model that allows interaction between the source code sequence modality and the graph modality in a deep and fine-grained way. We show that ProgSG reduces the RMSE of design performance predictions by up to $22%$, and identifies designs with an average of $1.10times$.
• Score: 45.471039079664656
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In recent years, domain-specific accelerators (DSAs) have gained popularity for applications such as deep learning and autonomous driving. To facilitate DSA designs, programmers use high-level synthesis (HLS) to compile a high-level description written in C/C++ into a design with low-level hardware description languages that eventually synthesize DSAs on circuits. However, creating a high-quality HLS design still demands significant domain knowledge, particularly in microarchitecture decisions expressed as \textit{pragmas}. Thus, it is desirable to automate such decisions with the help of machine learning for predicting the quality of HLS designs, requiring a deeper understanding of the program that consists of original code and pragmas. Naturally, these programs can be considered as sequence data. In addition, these programs can be compiled and converted into a control data flow graph (CDFG). But existing works either fail to leverage both modalities or combine the two in shallow or coarse ways. We propose ProgSG, a model that allows interaction between the source code sequence modality and the graph modality in a deep and fine-grained way. To alleviate the scarcity of labeled designs, a pre-training method is proposed based on a suite of compiler's data flow analysis tasks. Experimental results show that ProgSG reduces the RMSE of design performance predictions by up to $22\%$, and identifies designs with an average of $1.10\times$ and $1.26\times$ (up to $8.17\times$ and $13.31\times$) performance improvement in design space exploration (DSE) task compared to HARP and AutoDSE, respectively.

Related papers

• Hierarchical Mixture of Experts: Generalizable Learning for High-Level Synthesis [43.612837464039686]
  High-level synthesis (HLS) is a widely used tool in designing Field Programmable Gate Array (FPGA) We propose a more domain-generalizable model structure: a two-level hierarchical Mixture of Experts (MoE) In the low-level MoE, we apply MoE on three natural granularities of a program: node, basic block, and graph. The high-level MoE learns to aggregate the three granularities for the final decision.
  arXiv  Detail & Related papers  (2024-10-25T00:27:53Z)
• Learning to Compare Hardware Designs for High-Level Synthesis [44.408523725466374]
  High-level synthesis (HLS) is an automated design process that transforms high-level code into hardware designs. HLS relies on pragmas, which are directives inserted into the source code to guide the synthesis process. We propose compareXplore, a novel approach that learns to compare hardware designs for effective HLS optimization.
  arXiv  Detail & Related papers  (2024-09-20T00:47:29Z)
• Guess & Sketch: Language Model Guided Transpilation [59.02147255276078]
  Learned transpilation offers an alternative to manual re-writing and engineering efforts. Probabilistic neural language models (LMs) produce plausible outputs for every input, but do so at the cost of guaranteed correctness. Guess & Sketch extracts alignment and confidence information from features of the LM then passes it to a symbolic solver to resolve semantic equivalence.
  arXiv  Detail & Related papers  (2023-09-25T15:42:18Z)
• SEER: Super-Optimization Explorer for HLS using E-graph Rewriting with MLIR [0.3124884279860061]
  High-level synthesis (HLS) is a process that automatically translates a software program in a high-level language into a low-level hardware description. We propose a super-optimization approach for HLS that automatically rewrites an arbitrary software program into HLS efficient code. We show that SEER achieves up to 38x the performance within 1.4x the area of the original program.
  arXiv  Detail & Related papers  (2023-08-15T09:05:27Z)
• ProgSG: Cross-Modality Representation Learning for Programs in Electronic Design Automation [38.023395256208055]
  High-level synthesis (HLS) allows a developer to compile a high-level description in the form of software code in C and C++. HLS tools still require microarchitecture decisions, expressed in terms of pragmas. We propose ProgSG allowing the source code sequence modality and the graph modalities to interact with each other in a deep and fine-grained way.
  arXiv  Detail & Related papers  (2023-05-18T09:44:18Z)
• CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learning [92.36705236706678]
  "CodeRL" is a new framework for program synthesis tasks through pretrained LMs and deep reinforcement learning. During inference, we introduce a new generation procedure with a critical sampling strategy. For the model backbones, we extended the encoder-decoder architecture of CodeT5 with enhanced learning objectives.
  arXiv  Detail & Related papers  (2022-07-05T02:42:15Z)
• Matching Pursuit Based Scheduling for Over-the-Air Federated Learning [67.59503935237676]
  This paper develops a class of low-complexity device scheduling algorithms for over-the-air learning via the method of federated learning. Compared to the state-of-the-art proposed scheme, the proposed scheme poses a drastically lower efficiency system. The efficiency of the proposed scheme is confirmed via experiments on the CIFAR dataset.
  arXiv  Detail & Related papers  (2022-06-14T08:14:14Z)
• A Graph Deep Learning Framework for High-Level Synthesis Design Space Exploration [11.154086943903696]
  High-Level Synthesis is a solution for fast prototyping application-specific hardware. We propose HLS, for the first time in the literature, graph neural networks that jointly predict acceleration performance and hardware costs. We show that our approach achieves prediction accuracy comparable with that of commonly used simulators.
  arXiv  Detail & Related papers  (2021-11-29T18:17:45Z)
• Synthetic Datasets for Neural Program Synthesis [66.20924952964117]
  We propose a new methodology for controlling and evaluating the bias of synthetic data distributions over both programs and specifications. We demonstrate, using the Karel DSL and a small Calculator DSL, that training deep networks on these distributions leads to improved cross-distribution generalization performance.
  arXiv  Detail & Related papers  (2019-12-27T21:28:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.