LazyTensor: combining eager execution with domain-specific compilers

• URL: http://arxiv.org/abs/2102.13267v1
• Date: Fri, 26 Feb 2021 02:22:16 GMT
• Title: LazyTensor: combining eager execution with domain-specific compilers
• Abstract summary: LazyTensor is a technique to target domain compilers without sacrificing define-by-run ergonomics. Initially developed to support PyTorch on Cloud TPUs, along with a substantially shared implementation technique, has been used by Swift for specific accelerators.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Domain-specific optimizing compilers have demonstrated significant performance and portability benefits, but require programs to be represented in their specialized IRs. Existing frontends to these compilers suffer from the "language subset problem" where some host language features are unsupported in the subset of the user's program that interacts with the domain-specific compiler. By contrast, define-by-run ML frameworks-colloquially called "eager" mode-are popular due to their ease of use and expressivity, where the full power of the host programming language can be used. LazyTensor is a technique to target domain specific compilers without sacrificing define-by-run ergonomics. Initially developed to support PyTorch on Cloud TPUs, the technique, along with a substantially shared implementation, has been used by Swift for TensorFlow across CPUs, GPUs, and TPUs, demonstrating the generality of the approach across (1) Tensor implementations, (2) hardware accelerators, and (3) programming languages.

Related papers

• JLIR: A Julia-Native MLIR-Inspired Intermediate Representation with Automatic JACC Kernel Extraction [0.27998963147546135]
  We present JLIR (Julia-native Level Intermediate Representation), a Julia-native intermediate representation framework.<n>It brings the main benefits of MLIR-style multi-level, dialect-oriented compilation into the Julia ecosystem while remaining usable as ordinary Julia code.<n>It includes built-in dialects for arithmetic, control flow, functions, structured loops, and memory operations.
  arXiv  Detail & Related papers  (2026-09-04T00:35:54Z)
• Compile by Training: Turning Natural-Language Specifications into Local Neural Functions [8.34539885321864]
  compile by training turns a natural-language specification into a reusable neural function.<n>On FuzzyBench-Hard, a subset on which the Program-as-Weights fast compiler produced no exact matches, compile by training reaches 83.6% semantic accuracy.
  arXiv  Detail & Related papers  (2026-09-03T17:59:49Z)
• AgentCompile: An LLM-Guided Compiler for Direct CUDA Inference [58.285369364699]
  We present AgentCompile, an LLM-guided inference compiler that uses LLM outputs only as advisory search metadata.<n>Given compiler-derived region summaries and bounded candidate spaces, the LLM proposes semantic labels, candidate priorities, parameter hints, and risk annotations.<n>AgentCompile materializes candidates through graphs templates, checks interface and hardware constraints, validates candidates empirically, selects implementations by measured latency, and falls back when specialization is unsupported or unprofitable.<n>In end-to-end autoregressive generation, AgentCompile averages 5.66x, 4.05x, and 4.26x speedup over PyTorch on Qwen3-1.7B, Qwen
  arXiv  Detail & Related papers  (2026-06-04T03:49:56Z)
• A High-level Synthesis Toolchain for the Julia Language [1.7995266833057173]
  We propose a new MLIR-based compiler toolchain that unifies the development process by automatically compiling kernels written in the Julia programming language into SystemVerilog.<n>Our toolchain supports both dynamic and static scheduling, directly integrates with the AXI4-Stream protocol, and generates vendor-agnostic RTL.<n>This prototype toolchain is able to synthesize a set of signal processing/mathematical benchmarks that can operate at 100MHz on real FPGA devices.
  arXiv  Detail & Related papers  (2025-12-17T18:32:06Z)
• ATLAS: Automated Tree-based Language Analysis System for C and C++ source programs [1.0499611180329804]
  This paper introduces ATLAS, a Python-based Command-Line Interface (CLI) that generates statement-level Control Flow Graphs (CFG) and type-aware Data Flow Graphs (DFG)<n>ATLAS provides a practical foundation for improving downstream Software Engineering (SE) and machine-learning-based program understanding.
  arXiv  Detail & Related papers  (2025-12-14T01:11:11Z)
• Guided Tensor Lifting [54.10411390218929]
  Domain-specific languages (s) for machine learning are revolutionizing the speed and efficiency of machine learning workloads. To take advantage of these capabilities, a user must first translate their legacy code from the language it is currently written in, into the new DSL. Process of automatically lifting code into these DSLs has been identified by several recent works, which propose program synthesis as a solution.
  arXiv  Detail & Related papers  (2025-04-28T12:00:10Z)
• MSCCL++: Rethinking GPU Communication Abstractions for Cutting-edge AI Applications [9.28608047116196]
  Modern cutting-edge AI applications are being developed over fast-evolving, heterogeneous, nascent hardware devices. This requires frequent reworking of the AI software stack to adopt bottom-up changes from new hardware. We present MSCCL++, a novel abstraction of GPU communication based on separation concerns.
  arXiv  Detail & Related papers  (2025-04-11T23:51:54Z)
• An approach to performance portability through generic programming [0.0]
  This work describes a design approach that allows the integration of low-level and verbose programming tools into high-level generic algorithms based on template meta-programming in C++. That allows scientific software to be maintainable and efficient in a period of diversifying hardware in HPC.
  arXiv  Detail & Related papers  (2023-11-08T21:54:43Z)
• mlirSynth: Automatic, Retargetable Program Raising in Multi-Level IR using Program Synthesis [48.01697184432969]
  mlirSynth translates programs from lower-level MLIR dialects to high-level ones without manually defined rules. We demonstrate its effectiveness reviby raising C programs to two distinct high-level MLIR dialects, which enables us to use existing high-level dialect specific compilation flows.
  arXiv  Detail & Related papers  (2023-10-06T12:21:50Z)
• Harnessing Deep Learning and HPC Kernels via High-Level Loop and Tensor Abstractions on CPU Architectures [67.47328776279204]
  This work introduces a framework to develop efficient, portable Deep Learning and High Performance Computing kernels. We decompose the kernel development in two steps: 1) Expressing the computational core using Processing Primitives (TPPs) and 2) Expressing the logical loops around TPPs in a high-level, declarative fashion. We demonstrate the efficacy of our approach using standalone kernels and end-to-end workloads that outperform state-of-the-art implementations on diverse CPU platforms.
  arXiv  Detail & Related papers  (2023-04-25T05:04:44Z)
• QParallel: Explicit Parallelism for Programming Quantum Computers [62.10004571940546]
  We present a language extension for parallel quantum programming. QParallel removes ambiguities concerning parallelism in current quantum programming languages. We introduce a tool that guides programmers in the placement of parallel regions by identifying the subroutines that profit most from parallelization.
  arXiv  Detail & Related papers  (2022-10-07T16:35:16Z)
• Enabling Retargetable Optimizing Compilers for Quantum Accelerators via a Multi-Level Intermediate Representation [78.8942067357231]
  We present a multi-level quantum-classical intermediate representation (IR) that enables an optimizing, retargetable, ahead-of-time compiler. We support the entire gate-based OpenQASM 3 language and provide custom extensions for common quantum programming patterns and improved syntax. Our work results in compile times that are 1000x faster than standard Pythonic approaches, and 5-10x faster than comparative standalone quantum language compilers.
  arXiv  Detail & Related papers  (2021-09-01T17:29:47Z)
• StreamBlocks: A compiler for heterogeneous dataflow computing (technical report) [1.5293427903448022]
  This work introduces StreamBlocks, an open-source compiler and runtime that uses the CAL dataflow programming language to partition computations across platforms. StreamBlocks supports exploring the design space with a profile-guided tool that helps identify the best hardware-software partitions.
  arXiv  Detail & Related papers  (2021-07-20T08:46:47Z)
• Representing Partial Programs with Blended Abstract Semantics [62.20775388513027]
  We introduce a technique for representing partially written programs in a program synthesis engine. We learn an approximate execution model implemented as a modular neural network. We show that these hybrid neuro-symbolic representations enable execution-guided synthesizers to use more powerful language constructs.
  arXiv  Detail & Related papers  (2020-12-23T20:40:18Z)
• Extending C++ for Heterogeneous Quantum-Classical Computing [56.782064931823015]
  qcor is a language extension to C++ and compiler implementation that enables heterogeneous quantum-classical programming, compilation, and execution in a single-source context. Our work provides a first-of-its-kind C++ compiler enabling high-level quantum kernel (function) expression in a quantum-language manner.
  arXiv  Detail & Related papers  (2020-10-08T12:49:07Z)
• Instead of Rewriting Foreign Code for Machine Learning, Automatically Synthesize Fast Gradients [6.09170287691728]
  This paper presents Enzyme, a high-performance automatic differentiation (AD) compiler plugin for the LLVM compiler framework. Enzyme synthesizes gradients for programs written in any language whose compiler targets LLVM intermediate representation (IR) On a machine-learning focused benchmark suite including Microsoft's ADBench, AD on optimized IR achieves a geometric mean speedup of 4.5x over AD on IR.
  arXiv  Detail & Related papers  (2020-10-04T22:32:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.