The Deep Learning Compiler: A Comprehensive Survey
- URL: http://arxiv.org/abs/2002.03794v4
- Date: Fri, 28 Aug 2020 09:19:43 GMT
- Title: The Deep Learning Compiler: A Comprehensive Survey
- Authors: Mingzhen Li, Yi Liu, Xiaoyan Liu, Qingxiao Sun, Xin You, Hailong Yang,
Zhongzhi Luan, Lin Gan, Guangwen Yang, Depei Qian
- Abstract summary: We perform a comprehensive survey of existing DL compilers by dissecting the commonly adopted design in details.
Specifically, we provide a comprehensive comparison among existing DL compilers from various aspects.
- Score: 16.19025439622745
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The difficulty of deploying various deep learning (DL) models on diverse DL
hardware has boosted the research and development of DL compilers in the
community. Several DL compilers have been proposed from both industry and
academia such as Tensorflow XLA and TVM. Similarly, the DL compilers take the
DL models described in different DL frameworks as input, and then generate
optimized codes for diverse DL hardware as output. However, none of the
existing survey has analyzed the unique design architecture of the DL compilers
comprehensively. In this paper, we perform a comprehensive survey of existing
DL compilers by dissecting the commonly adopted design in details, with
emphasis on the DL oriented multi-level IRs, and frontend/backend
optimizations. Specifically, we provide a comprehensive comparison among
existing DL compilers from various aspects. In addition, we present detailed
analysis on the design of multi-level IRs and illustrate the commonly adopted
optimization techniques. Finally, several insights are highlighted as the
potential research directions of DL compiler. This is the first survey paper
focusing on the design architecture of DL compilers, which we hope can pave the
road for future research towards DL compiler.
Related papers
- A Tale of Two DL Cities: When Library Tests Meet Compiler [12.751626834965231]
We propose OPERA to extract domain knowledge from the test inputs for DL libraries.
OPERA constructs diverse tests from the various test inputs for DL libraries.
It incorporates a diversity-based test prioritization strategy to migrate and execute those test inputs.
arXiv Detail & Related papers (2024-07-23T16:35:45Z) - What's Wrong with Your Code Generated by Large Language Models? An Extensive Study [80.18342600996601]
Large language models (LLMs) produce code that is shorter yet more complicated as compared to canonical solutions.
We develop a taxonomy of bugs for incorrect codes that includes three categories and 12 sub-categories, and analyze the root cause for common bug types.
We propose a novel training-free iterative method that introduces self-critique, enabling LLMs to critique and correct their generated code based on bug types and compiler feedback.
arXiv Detail & Related papers (2024-07-08T17:27:17Z) - IMDL-BenCo: A Comprehensive Benchmark and Codebase for Image Manipulation Detection & Localization [58.32394109377374]
IMDL-BenCo is the first comprehensive IMDL benchmark and modular.
It decomposes the IMDL framework into standardized, reusable components and revises the model construction pipeline.
It includes 8 state-of-the-art IMDL models (1 of which are reproduced from scratch), 2 sets of standard training and evaluation protocols, 15 GPU-accelerated evaluation metrics, and 3 kinds of robustness evaluation.
arXiv Detail & Related papers (2024-06-15T09:44:54Z) - A Survey of Deep Learning Library Testing Methods [33.62859142913532]
Deep learning (DL) libraries undertake the underlying optimization and computation.
DL libraries are not immune to bugs, which can pose serious threats to users' personal property and safety.
This paper provides an overview of the testing research related to various DL libraries.
arXiv Detail & Related papers (2024-04-27T11:42:13Z) - Serving Deep Learning Model in Relational Databases [72.72372281808694]
Serving deep learning (DL) models on relational data has become a critical requirement across diverse commercial and scientific domains.
We highlight three pivotal paradigms: The state-of-the-artDL-Centric architecture offloadsDL computations to dedicated DL frameworks.
The potential UDF-Centric architecture encapsulates one or more tensor computations into User Defined Functions (UDFs) within the database system.
The potentialRelation-Centric architecture aims to represent a large-scale tensor computation through operators.
arXiv Detail & Related papers (2023-10-07T06:01:35Z) - A Comprehensive Benchmark of Deep Learning Libraries on Mobile Devices [12.342282138576348]
We build a benchmark that includes 6 representative DL libs and 15 diversified DL models.
We then perform extensive experiments on 10 mobile devices, which help reveal a complete landscape of the current mobile DL libs ecosystem.
We find that the best-performing DL lib is severely fragmented across different models and hardware.
arXiv Detail & Related papers (2022-02-14T07:00:31Z) - Design Smells in Deep Learning Programs: An Empirical Study [9.112172220055431]
Design smells in Deep Learning (DL) programs are poor design and-or configuration decisions taken during the development of DL components.
We present a catalogue of 8 design smells for a popular DL architecture, namely deep Feedforward Neural Networks.
arXiv Detail & Related papers (2021-07-05T21:26:05Z) - Tensor Processing Primitives: A Programming Abstraction for Efficiency
and Portability in Deep Learning Workloads [86.62083829086393]
This work introduces the Processing Primitives (TPP), a programming abstraction striving for efficient, portable implementation of Deep Learning-workloads with high-productivity.
TPPs define a compact, yet versatile set of 2D-tensor operators (or a virtual ISA), which can be utilized as building-blocks to construct complex operators on high-dimensional tensors.
We demonstrate the efficacy of our approach using standalone kernels and end-to-end DL-workloads expressed entirely via TPPs that outperform state-of-the-art implementations on multiple platforms.
arXiv Detail & Related papers (2021-04-12T18:35:49Z) - A Survey of Deep Active Learning [54.376820959917005]
Active learning (AL) attempts to maximize the performance gain of the model by marking the fewest samples.
Deep learning (DL) is greedy for data and requires a large amount of data supply to optimize massive parameters.
Deep active learning (DAL) has emerged.
arXiv Detail & Related papers (2020-08-30T04:28:31Z) - PolyDL: Polyhedral Optimizations for Creation of High Performance DL
primitives [55.79741270235602]
We present compiler algorithms to automatically generate high performance implementations of Deep Learning primitives.
We develop novel data reuse analysis algorithms using the polyhedral model.
We also show that such a hybrid compiler plus a minimal library-use approach results in state-of-the-art performance.
arXiv Detail & Related papers (2020-06-02T06:44:09Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.