CUDA-Harness: Harnessing Agentic CUDA Kernel Generation and Optimization from Natural Language

• URL: http://arxiv.org/abs/2609.00058v1
• Date: Sun, 30 Aug 2026 13:51:43 GMT
• Title: CUDA-Harness: Harnessing Agentic CUDA Kernel Generation and Optimization from Natural Language
• Abstract summary: Text2CUDA makes generating kernels directly from natural language (Text2CUDA) essential.<n> Large Language Models (LLMs) prompts a series of works exploring LLM-based kernel generation.<n>We propose a framework for harnessing agentic kernel-level optimization from natural language.
• Score: 12.536212788514815
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Developing high-performance CUDA kernels demands specialized knowledge in algorithm implementation, correctness validation, and hardware-aware parallel optimization, creating a substantial expertise barrier and making generating CUDA kernels directly from natural language (Text2CUDA) essential. Meanwhile, the general-purpose code generation capability of Large Language Models (LLMs) prompts a series of works exploring LLM-based CUDA kernel generation. They mainly focus on transpilation from high-level frameworks such as PyTorch to CUDA (Torch2CUDA) rather than Text2CUDA, where models must understand the high-level input semantics and handle low-level kernel implementation and validation. Additionally, these methods are vulnerable to reward hacking due to reliance on predefined test inputs. In this paper, we propose CUDA-Harness, a framework for harnessing agentic CUDA kernel generation and optimization from natural language. Specifically, we introduce Intermediate-Structured Generation to connect high-level semantic understanding with low-level kernel generation. To dilute reward hacking in Text2CUDA, we construct Synthesis-Based Verification to provide isolated test data and progressive validation. Furthermore, we propose Feedback-Adaptive Evolution, a kernel evolution strategy that prioritizes correctness while optimizing performance. Finally, through extensive experiments, we demonstrate the effectiveness of CUDA-Harness, with further evaluations illustrating generalization across LLMs, hardware platforms, and to C-to-CUDA transpilation.

Related papers

• Accelerated Decoding of Centroid Positional Encoding for Instance Segmentation [49.97373153077311]
  We investigate the decoding overhead associated with a centroid encoding for Instance, in which each pixel regresses a positional embedding of its instance centroid.<n>Our solution significantly reduces decoding overhead and improves Endto-End latency, outperforming both CPU-based approaches and naive GPU implementations.
  arXiv  Detail & Related papers  (2026-09-15T09:05:17Z)
• GPU Forecasters: Language Models as Selective Surrogates for Kernel Runtime Optimization [73.92934811090163]
  We study how LLMs can serve as selective GPU surrogates for kernel evaluation.<n>We measure whether their forecasts are accurate, calibrated, and practically useful for recovering fast kernels under limited GPU-measurement budgets.<n>Our experiments demonstrate that LLMs can accurately forecast relative kernel performance, that their utility can be improved through reinforcement learning.
  arXiv  Detail & Related papers  (2026-05-29T15:56:08Z)
• CUDAHercules: Benchmarking Hardware-Aware Expert-level CUDA Optimization for LLMs [45.52512820646709]
  We introduce Hercules, a benchmark that evaluates generated against end-to-end human-expert SOTA systems.<n>We show that automated programming remains far from fully solved and requires stronger hardware reasoning, better tool use, and training objectives.
  arXiv  Detail & Related papers  (2026-05-08T20:35:27Z)
• CUDA Agent: Large-Scale Agentic RL for High-Performance CUDA Kernel Generation [51.72529978689561]
  Agent is a large-scale agentic reinforcement learning system that develops kernel expertise through three components.<n>Agent delivers 100%, 100%, and 92% faster rate over torchcompile on KernelBench.
  arXiv  Detail & Related papers  (2026-02-27T18:58:05Z)
• KernelBlaster: Continual Cross-Task CUDA Optimization via Memory-Augmented In-Context Reinforcement Learning [3.4998382481249286]
  We release KernelBlaster as an open-source agentic framework, accompanied by a test harness, verification components, and a reproducible evaluation.<n>Our method achieves mean speed-ups of 1.43x, 2.50x, and 1.50x on KernelBench Levels 1, 2, and 3, respectively.
  arXiv  Detail & Related papers  (2026-02-15T19:48:43Z)
• CUDABench: Benchmarking LLMs for Text-to-CUDA Generation [11.395567633975878]
  Recent studies have demonstrated the potential of Large Language Models (LLMs) in generating GPUs.<n>Given the hardware-specific and performance-critical features of programming, accurately assessing the performance of LLM-generated GPU programs is nontrivial.<n>We introduce GPUBench, a comprehensive benchmark designed to evaluate the text-to-CUDA capabilities of LLMs.
  arXiv  Detail & Related papers  (2026-02-13T09:46:45Z)
• DICE: Diffusion Large Language Models Excel at Generating CUDA Kernels [17.979042914049842]
  Diffusion large language models (dLLMs) have emerged as a compelling alternative to autoregressive (AR) LLMs.<n>CuKe is an augmented dataset optimized for high-performance kernels.<n>DICE is a series of diffusion large language models designed for kernel generation.
  arXiv  Detail & Related papers  (2026-02-12T08:45:13Z)
• Towards Robust Agentic CUDA Kernel Benchmarking, Verification, and Optimization [25.135006275638172]
  We introduce robust-kbench, a new benchmark for rigorous evaluation of kernel performance and correctness across varied scenarios.<n>We also present a comprehensive agentic framework that translates torch code to kernels and iteratively improve their runtime setting.<n>Our approach produces kernels outperforming torch implementations for practical applications, including forward and backward passes.
  arXiv  Detail & Related papers  (2025-09-16T11:08:30Z)
• Harnessing Deep Learning and HPC Kernels via High-Level Loop and Tensor Abstractions on CPU Architectures [67.47328776279204]
  This work introduces a framework to develop efficient, portable Deep Learning and High Performance Computing kernels. We decompose the kernel development in two steps: 1) Expressing the computational core using Processing Primitives (TPPs) and 2) Expressing the logical loops around TPPs in a high-level, declarative fashion. We demonstrate the efficacy of our approach using standalone kernels and end-to-end workloads that outperform state-of-the-art implementations on diverse CPU platforms.
  arXiv  Detail & Related papers  (2023-04-25T05:04:44Z)
• PolyScientist: Automatic Loop Transformations Combined with Microkernels for Optimization of Deep Learning Primitives [55.79741270235602]
  We develop a hybrid solution to the development of deep learning kernels. We use the advanced polyhedral technology to automatically tune the outer loops for performance.
  arXiv  Detail & Related papers  (2020-02-06T08:02:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.