Integrating Performance Tools in Model Reasoning for GPU Kernel Optimization

• URL: http://arxiv.org/abs/2510.17158v1
• Date: Mon, 20 Oct 2025 04:57:50 GMT
• Title: Integrating Performance Tools in Model Reasoning for GPU Kernel Optimization
• Authors: Daniel Nichols, Konstantinos Parasyris, Charles Jekel, Abhinav Bhatele, Harshitha Menon,
• Abstract summary: We propose a methodology to train language models that can interact with performance tools.<n>We then demonstrate how this methodology can be used to train a state-of-the-art GPU kernel optimization model.
• Score: 4.086371789129554
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Language models are now prevalent in software engineering with many developers using them to automate tasks and accelerate their development. While language models have been tremendous at accomplishing complex software engineering tasks, there are still many areas where they fail to deliver desirable results, for instance code performance related tasks. Tasks like optimization depend on many complex data from the environment, hardware, etc. that are not directly represented in source code. Recent efforts have seen large improvements in general code modeling tasks using chain-of-thought style reasoning, but these models still fail to comprehend how the environment interacts with code performance. In this paper we propose a methodology to train language models that can interact with performance tools during their reasoning process. We then demonstrate how this methodology can be used to train a state-of-the-art GPU kernel optimization model.

Related papers

• Darkit: A User-Friendly Software Toolkit for Spiking Large Language Model [50.37090759139591]
  Large language models (LLMs) have been widely applied in various practical applications, typically comprising billions of parameters.<n>The human brain, employing bio-plausible spiking mechanisms, can accomplish the same tasks while significantly reducing energy consumption.<n>We are releasing a software toolkit named DarwinKit (Darkit) to accelerate the adoption of brain-inspired large language models.
  arXiv  Detail & Related papers  (2024-12-20T07:50:08Z)
• Optimizing Large Language Models for OpenAPI Code Completion [0.0]
  This study evaluates the OpenAPI completion performance of GitHub Copilot. It proposes a set of task-specific optimizations leveraging Meta's open-source model Code Llama. The fine-tuned Code Llama model reaches a peak correctness improvement of 55.2% over GitHub Copilot.
  arXiv  Detail & Related papers  (2024-05-24T17:19:03Z)
• Can Large Language Models Write Parallel Code? [0.5317767988097261]
  Large language models are increasingly becoming a popular tool for software development. In this paper, we study the capabilities of state-of-the-art language models to generate parallel code.
  arXiv  Detail & Related papers  (2024-01-23T08:25:12Z)
• CoLLiE: Collaborative Training of Large Language Models in an Efficient Way [59.09824823710863]
  CoLLiE is an efficient library that facilitates collaborative training of large language models. With its modular design and comprehensive functionality, CoLLiE offers a balanced blend of efficiency, ease of use, and customization.
  arXiv  Detail & Related papers  (2023-12-01T08:02:16Z)
• HPC-Coder: Modeling Parallel Programs using Large Language Models [2.3101915391170573]
  We show how large language models can be applied to tasks specific to high performance and scientific codes. We introduce a new dataset of HPC and scientific codes and use it to fine-tune several pre-trained models. In our experiments, we show that this model can auto-complete HPC functions where generic models cannot.
  arXiv  Detail & Related papers  (2023-06-29T19:44:55Z)
• A Static Evaluation of Code Completion by Large Language Models [65.18008807383816]
  Execution-based benchmarks have been proposed to evaluate functional correctness of model-generated code on simple programming problems. static analysis tools such as linters, which can detect errors without running the program, haven't been well explored for evaluating code generation models. We propose a static evaluation framework to quantify static errors in Python code completions, by leveraging Abstract Syntax Trees.
  arXiv  Detail & Related papers  (2023-06-05T19:23:34Z)
• Learning Performance-Improving Code Edits [107.21538852090208]
  We introduce a framework for adapting large language models (LLMs) to high-level program optimization. First, we curate a dataset of performance-improving edits made by human programmers of over 77,000 competitive C++ programming submission pairs. For prompting, we propose retrieval-based few-shot prompting and chain-of-thought, and for finetuning, these include performance-conditioned generation and synthetic data augmentation based on self-play.
  arXiv  Detail & Related papers  (2023-02-15T18:59:21Z)
• Cramming: Training a Language Model on a Single GPU in One Day [64.18297923419627]
  Recent trends in language modeling have focused on increasing performance through scaling. We investigate the downstream performance achievable with a transformer-based language model trained completely from scratch with masked language modeling for a single day on a single consumer GPU. We provide evidence that even in this constrained setting, performance closely follows scaling laws observed in large-compute settings.
  arXiv  Detail & Related papers  (2022-12-28T18:59:28Z)
• Top-KAST: Top-K Always Sparse Training [50.05611544535801]
  We propose Top-KAST, a method that preserves constant sparsity throughout training. We show that it performs comparably to or better than previous works when training models on the established ImageNet benchmark. In addition to our ImageNet results, we also demonstrate our approach in the domain of language modeling.
  arXiv  Detail & Related papers  (2021-06-07T11:13:05Z)
• Paraphrastic Representations at Scale [134.41025103489224]
  We release trained models for English, Arabic, German, French, Spanish, Russian, Turkish, and Chinese languages. We train these models on large amounts of data, achieving significantly improved performance from the original papers.
  arXiv  Detail & Related papers  (2021-04-30T16:55:28Z)
• A Learned Performance Model for Tensor Processing Units [5.733911161090224]
  We demonstrate a method of learning performance models from a corpus of graph programs for Processing Unit (TPU) instances. We show that our learned model outperforms a heavily-optimized analytical performance model on two tasks. It helps an autotuner discover faster programs in a setting where access to TPUs is limited or expensive.
  arXiv  Detail & Related papers  (2020-08-03T17:24:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.