Learning to Make Compiler Optimizations More Effective

• URL: http://arxiv.org/abs/2102.13514v1
• Date: Wed, 24 Feb 2021 10:42:56 GMT
• Title: Learning to Make Compiler Optimizations More Effective
• Authors: Rahim Mammadli, Marija Selakovic, Felix Wolf, Michael Pradel
• Abstract summary: LoopLearner predicts which way of writing a loop will lead to efficient compiled code. We evaluate LoopLearner with 1,895 loops from various performance-relevant benchmarks.
• Score: 11.125012960514471
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Because loops execute their body many times, compiler developers place much emphasis on their optimization. Nevertheless, in view of highly diverse source code and hardware, compilers still struggle to produce optimal target code. The sheer number of possible loop optimizations, including their combinations, exacerbates the problem further. Today's compilers use hard-coded heuristics to decide when, whether, and which of a limited set of optimizations to apply. Often, this leads to highly unstable behavior, making the success of compiler optimizations dependent on the precise way a loop has been written. This paper presents LoopLearner, which addresses the problem of compiler instability by predicting which way of writing a loop will lead to efficient compiled code. To this end, we train a neural network to find semantically invariant source-level transformations for loops that help the compiler generate more efficient code. Our model learns to extract useful features from the raw source code and predicts the speedup that a given transformation is likely to yield. We evaluate LoopLearner with 1,895 loops from various performance-relevant benchmarks. Applying the transformations that our model deems most favorable prior to compilation yields an average speedup of 1.14x. When trying the top-3 suggested transformations, the average speedup even increases to 1.29x. Comparing the approach with an exhaustive search through all available code transformations shows that LoopLearner helps to identify the most beneficial transformations in several orders of magnitude less time.

Related papers

• Should AI Optimize Your Code? A Comparative Study of Current Large Language Models Versus Classical Optimizing Compilers [0.0]
  Large Language Models (LLMs) raise intriguing questions about the potential for AI-driven approaches to revolutionize code optimization methodologies. This paper presents a comparative analysis between two state-of-the-art Large Language Models, GPT-4.0 and CodeLlama-70B, and traditional optimizing compilers.
  arXiv  Detail & Related papers  (2024-06-17T23:26:41Z)
• CompilerDream: Learning a Compiler World Model for General Code Optimization [58.87557583347996]
  We introduce CompilerDream, a model-based reinforcement learning approach to general code optimization. It comprises a compiler world model that accurately simulates the intrinsic properties of optimization passes and an agent trained on this model to produce effective optimization strategies. It excels across diverse datasets, surpassing LLVM's built-in optimizations and other state-of-the-art methods in both settings of value prediction and end-to-end code optimization.
  arXiv  Detail & Related papers  (2024-04-24T09:20:33Z)
• Large Language Models for Compiler Optimization [22.52765975286403]
  We present a transformer model trained from scratch to optimize LLVM assembly for code size. We ask the model to predict the instruction counts before and after optimization, and the optimized code itself. Our approach achieves a 3.0% improvement in reducing instruction counts over the compiler.
  arXiv  Detail & Related papers  (2023-09-11T22:11:46Z)
• Quantum Circuit Optimization and Transpilation via Parameterized Circuit Instantiation [0.0]
  We describe algorithms to apply instantiation during two common compilation steps: circuit optimization and gate-set transpilation. Our circuit optimization algorithm produces circuits with an average of 13% fewer gates than other optimizing compilers. Our gate-set transpilation algorithm can target any gate-set, even sets with multiple two-qubit gates, and produces circuits with an average of 12% fewer two-qubit gates than other compilers.
  arXiv  Detail & Related papers  (2022-06-16T02:22:08Z)
• Will Bilevel Optimizers Benefit from Loops [63.22466953441521]
  Two current popular bilevelimats AID-BiO and ITD-BiO naturally involve solving one or two sub-problems. We first establish unified convergence analysis for both AID-BiO and ITD-BiO that are applicable to all implementation choices of loops.
  arXiv  Detail & Related papers  (2022-05-27T20:28:52Z)
• LoopStack: a Lightweight Tensor Algebra Compiler Stack [61.04098601022665]
  LoopStack is a domain specific compiler stack for tensor operations. It generates machine code that matches and frequently exceeds the performance of in state-of-the-art machine learning frameworks. It has a very small memory footprint - a binary size of 245KB, and under 30K lines of effective code makes it ideal for use on mobile and embedded devices.
  arXiv  Detail & Related papers  (2022-05-02T01:57:58Z)
• A Fully Single Loop Algorithm for Bilevel Optimization without Hessian Inverse [121.54116938140754]
  We propose a new Hessian inverse free Fully Single Loop Algorithm for bilevel optimization problems. We show that our algorithm converges with the rate of $O(epsilon-2)$.
  arXiv  Detail & Related papers  (2021-12-09T02:27:52Z)
• Autotuning Search Space for Loop Transformations [0.03683202928838612]
  We propose a loop transformation search space that takes the form of a tree. We implemented a simple autotuner exploring the search space and applied it to a selected set of PolyBench kernels.
  arXiv  Detail & Related papers  (2020-10-13T16:26:57Z)
• Static Neural Compiler Optimization via Deep Reinforcement Learning [1.458855293397494]
  In this paper, we employ a deep reinforcement learning approach to the phase-ordering problem. Provided with sub-sequences constituting LLVM's O3 sequence, our agent learns to outperform the O3 sequence on the set of source codes used for training. We believe that the models trained using our approach can be integrated into modern compilers as neural optimization agents.
  arXiv  Detail & Related papers  (2020-08-20T13:16:29Z)
• Kernel methods through the roof: handling billions of points efficiently [94.31450736250918]
  Kernel methods provide an elegant and principled approach to nonparametric learning, but so far could hardly be used in large scale problems. Recent advances have shown the benefits of a number of algorithmic ideas, for example combining optimization, numerical linear algebra and random projections. Here, we push these efforts further to develop and test a solver that takes full advantage of GPU hardware.
  arXiv  Detail & Related papers  (2020-06-18T08:16:25Z)
• PolyDL: Polyhedral Optimizations for Creation of High Performance DL primitives [55.79741270235602]
  We present compiler algorithms to automatically generate high performance implementations of Deep Learning primitives. We develop novel data reuse analysis algorithms using the polyhedral model. We also show that such a hybrid compiler plus a minimal library-use approach results in state-of-the-art performance.
  arXiv  Detail & Related papers  (2020-06-02T06:44:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.