Harness Engineering for LLM-Driven GPU Kernel Generation

• URL: http://arxiv.org/abs/2607.17979v1
• Date: Mon, 20 Jul 2026 14:14:14 GMT
• Title: Harness Engineering for LLM-Driven GPU Kernel Generation
• Abstract summary: Large language models (LLMs) can assist GPU kernel generation, but their practical effectiveness depends on whether generated code can be reliably constrained, validated, profiled, and selected.<n>This paper presents a harness-centered system for LLM-driven GPU kernel optimization in the MLSys 2026 FlashInfer AI Kernel Generation Contest.
• Score: 1.375573703671189
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) can assist GPU kernel generation, but their practical effectiveness depends on whether generated code can be reliably constrained, validated, profiled, and selected. This paper presents a harness-centered system for LLM-driven GPU kernel optimization in the MLSys 2026 FlashInfer AI Kernel Generation Contest on NVIDIA Blackwell B200 GPUs. The system separates an evaluation harness from a profile-backed optimization controller: the harness enforces compilation, correctness, official-aligned timing, and artifact archival, while the controller turns profiler and workload evidence into bounded candidate-generation decisions. Human-authored skills capture operator constraints, references, profiling procedures, and promotion rules, while Codex and Claude Code agents generate candidate kernels inside those constraints. Across five operator definitions, the retained official-aligned artifacts achieved mean-latency speedups over supplied FlashInfer baselines of 1.62x, 18.05x, 29.68x, 1.12x, and 13.70x. The Agent-Assisted kernels outperform the Full-Agent artifacts across the evaluated definitions, indicating that expert-provided optimization directions, high-quality references, and workload context remain critical for reliable AI-driven kernel optimization.

Related papers

• Are LLM-Generated GPU Kernels Production-Ready? A Trace-Driven Benchmark and Optimization Agent [10.241408048987038]
  Existing GPU kernel generation benchmarks draw problems from synthetic or curated sources that diverge from deployed workloads.<n>We present Atrex-Bench, a benchmark whose 30 operators and 440 shapes are sampled directly from full-cluster production inference traces of compute-limited, memory-rich GPUs.
  arXiv  Detail & Related papers  (2026-07-16T03:49:50Z)
• GPU Forecasters: Language Models as Selective Surrogates for Kernel Runtime Optimization [73.92934811090163]
  We study how LLMs can serve as selective GPU surrogates for kernel evaluation.<n>We measure whether their forecasts are accurate, calibrated, and practically useful for recovering fast kernels under limited GPU-measurement budgets.<n>Our experiments demonstrate that LLMs can accurately forecast relative kernel performance, that their utility can be improved through reinforcement learning.
  arXiv  Detail & Related papers  (2026-05-29T15:56:08Z)
• FastKernels: Benchmarking GPU Kernel Generation in Production [15.284289018908781]
  LLM-based agents for GPU kernel generation are advancing rapidly.<n>Yet their progress is constrained by the benchmarks they optimize against.<n>Fast Kernels is a production-grade inference framework that runs at parity with hardened systems.
  arXiv  Detail & Related papers  (2026-05-22T04:19:04Z)
• ARGUS: Agentic GPU Optimization Guided by Data-Flow Invariants [12.49256588033198]
  LLM-based coding agents can generate functionally correct GPU kernels, yet their performance remains far below hand-optimized libraries on critical computations.<n>We present Argus, an agentic framework that addresses this through data-flow invariants.<n>We evaluate Argus on the AMD MI300X GPU across GEMM, flash attention, and MoE kernels.
  arXiv  Detail & Related papers  (2026-04-16T15:49:31Z)
• Kernel-Smith: A Unified Recipe for Evolutionary Kernel Optimization [48.656549870801285]
  Kernel-Smith is a framework for high-performance GPU kernel and operator generation.<n>On the agent side, Kernel-Smith maintains a population of executable candidates and iteratively improves them.<n>On the training side, we convert long-horizon evolution trajectories into step-centric supervision and reinforcement learning signals.
  arXiv  Detail & Related papers  (2026-03-30T12:12:49Z)
• SOL-ExecBench: Speed-of-Light Benchmarking for Real-World GPU Kernels Against Hardware Limits [33.140868197031985]
  SOL-ExecBench is a benchmark of 235 optimization problems extracted from 124 production and emerging AI models.<n>It measures performance against analytically derived Speed-of-Light (SOL) bounds computed by SOLAR.<n>We report a SOL Score that quantifies how much of the gap between a release-defined scoring baseline and the hardware SOL bound a candidate kernel closes.
  arXiv  Detail & Related papers  (2026-03-19T17:30:02Z)
• CUDA Agent: Large-Scale Agentic RL for High-Performance CUDA Kernel Generation [51.72529978689561]
  Agent is a large-scale agentic reinforcement learning system that develops kernel expertise through three components.<n>Agent delivers 100%, 100%, and 92% faster rate over torchcompile on KernelBench.
  arXiv  Detail & Related papers  (2026-02-27T18:58:05Z)
• Outrunning LLM Cutoffs: A Live Kernel Crash Resolution Benchmark for All [57.23434868678603]
  Live-kBench is an evaluation framework for self-evolving benchmarks that scrapes and evaluates agents on freshly discovered kernel bugs.<n> kEnv is an agent-agnostic crash-resolution environment for kernel compilation, execution, and feedback.<n>Using kEnv, we benchmark three state-of-the-art agents, showing that they resolve 74% of crashes on the first attempt.
  arXiv  Detail & Related papers  (2026-02-02T19:06:15Z)
• PRAGMA: A Profiling-Reasoned Multi-Agent Framework for Automatic Kernel Optimization [12.24680414520151]
  PRAGMA is a profile-guided AI kernel generation framework.<n>It integrates execution feedback and fine-grained hardware profiling into the reasoning loop.<n>We evaluate PRAGMA on KernelBench, covering GPU and CPU backends.
  arXiv  Detail & Related papers  (2025-11-09T12:01:43Z)
• Harnessing Deep Learning and HPC Kernels via High-Level Loop and Tensor Abstractions on CPU Architectures [67.47328776279204]
  This work introduces a framework to develop efficient, portable Deep Learning and High Performance Computing kernels. We decompose the kernel development in two steps: 1) Expressing the computational core using Processing Primitives (TPPs) and 2) Expressing the logical loops around TPPs in a high-level, declarative fashion. We demonstrate the efficacy of our approach using standalone kernels and end-to-end workloads that outperform state-of-the-art implementations on diverse CPU platforms.
  arXiv  Detail & Related papers  (2023-04-25T05:04:44Z)
• PolyScientist: Automatic Loop Transformations Combined with Microkernels for Optimization of Deep Learning Primitives [55.79741270235602]
  We develop a hybrid solution to the development of deep learning kernels. We use the advanced polyhedral technology to automatically tune the outer loops for performance.
  arXiv  Detail & Related papers  (2020-02-06T08:02:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.