Compiler-Grounded Hierarchical Diagnosis for LLM-Based Triton Kernel Optimization

• URL: http://arxiv.org/abs/2607.23089v1
• Date: Sat, 25 Jul 2026 07:49:38 GMT
• Title: Compiler-Grounded Hierarchical Diagnosis for LLM-Based Triton Kernel Optimization
• Abstract summary: We present a compiler-grounded and hierarchical optimization framework for Triton kernels.<n>The system attains a geometric-mean speedup of 4.35$times$ and a median speedup of 2.73$times$ from the initial optimized to Triton kernel.<n>The complete distribution ranges from near-baseline entries to large wins, motivating transparent reporting of the current system's scope and limitations.
• Score: 10.855607995195099
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advances in large language models (LLMs) have enabled automated kernel generation and optimization, but most existing approaches rely on surface signals such as compilation feedback and profiling metrics. These signals reveal that a kernel is slow, but not why the backend compiler fails to realize a profitable optimization, especially on emerging accelerators such as NPUs. We therefore formulate kernel optimization as a progressive cross-layer diagnosis problem that links runtime symptoms to IR structure and compiler behavior before rewriting source. Based on this insight, we present our system, a compiler-grounded and hierarchical optimization framework for Triton kernels. the system escalates from lightweight pattern triage and profiling diagnosis to IR attribution and compiler-grounded analysis only when deeper evidence is needed, then proposes evidence-backed source-level rewrites. We implement the system on Triton for Ascend NPUs and evaluate it on 37 successfully converted entries from a standardized NPUKernelBench-derived Ascend 950 benchmark. Across these entries, the system attains a geometric-mean speedup of 4.35$\times$ and a median speedup of 2.73$\times$ from the initial to optimized Triton kernel; 22/37 exceed 2$\times$ and 13/37 exceed 5$\times$. The complete distribution ranges from near-baseline entries to large wins, motivating transparent reporting of the current system's scope and limitations.

Related papers

• Optimizing CUDA like a Human: Micro-Profiling Tools as Expert Surrogates for LLM-Based GPU Kernel Optimization [37.645358922637065]
  KernelPro is a closed-loop multi-agent system that automatically generates, profiles, and iteratively optimize kernel code.<n>On Kernel, KernelPro achieves mean speedups of 2.42x/4.69x/5.30x on Levels 1/2/3, establishing state-of-the-art performance across all difficulty levels.
  arXiv  Detail & Related papers  (2026-06-24T23:28:09Z)
• optimize_anything: A Universal API for Optimizing any Text Parameter [98.42497715725356]
  We show a single AI-based optimization system-supporting single-task search, multi-task search with cross-problem transfer, and generalization to unseen inputs.<n>We show for the first time that text optimization with LLM-based search is a general-purpose problem-solving paradigm.
  arXiv  Detail & Related papers  (2026-05-19T10:18:12Z)
• AgentKernelArena: Generalization-Aware Benchmarking of GPU Kernel Optimization Agents [30.03012293068346]
  AgentArena is an open-source benchmark for measuring AI coding agents on GPU kernel optimization.<n>We find near-perfect compilation and high correctness rates on most task categories.<n>Our unseen-configuration evaluation shows that HIP-to-HIP and Triton-to-Triton optimizations largely transfer to unseen input shapes.
  arXiv  Detail & Related papers  (2026-05-16T05:25:11Z)
• Diff3R: Feed-forward 3D Gaussian Splatting with Uncertainty-aware Differentiable Optimization [76.38917994186733]
  Diff3R is a novel framework that bridges feed-forward prediction and test-time optimization.<n>We show that it can be seamlessly integrated into existing feed-forward 3DGS architectures for both pose-given and pose-free methods.
  arXiv  Detail & Related papers  (2026-04-01T15:40:20Z)
• Improving Efficiency of GPU Kernel Optimization Agents using a Domain-Specific Language and Speed-of-Light Guidance [2.6106833542109693]
  We make two key observations about the abstraction level that agents operate at.<n>We implement these principles in $$CUTLASS, a DSL with a compiler for CUTLASS-backed GPU kernels.<n>We use SOL guidance to estimate headroom and guide optimization trials, deprioritize problems that are near SOL, and flag kernels that game the benchmark.
  arXiv  Detail & Related papers  (2026-03-30T21:16:39Z)
• Kernel-Smith: A Unified Recipe for Evolutionary Kernel Optimization [48.656549870801285]
  Kernel-Smith is a framework for high-performance GPU kernel and operator generation.<n>On the agent side, Kernel-Smith maintains a population of executable candidates and iteratively improves them.<n>On the training side, we convert long-horizon evolution trajectories into step-centric supervision and reinforcement learning signals.
  arXiv  Detail & Related papers  (2026-03-30T12:12:49Z)
• Towards Cold-Start Drafting and Continual Refining: A Value-Driven Memory Approach with Application to NPU Kernel Synthesis [68.7701048879757]
  EvoKernel is a self-evolving agentic framework that automates the lifecycle of kernel synthesis.<n>It learns stage-specific Q-values that prioritize experiences based on their contribution to the current objective.<n>It improves models' correctness from 11.0% to 83.0% and a median speedup of 3.60x over initial drafts.
  arXiv  Detail & Related papers  (2026-03-11T14:57:06Z)
• A Two-Stage GPU Kernel Tuner Combining Semantic Refactoring and Search-Based Optimization [9.49293344824955]
  This paper introduces a template-based rewriting layer on top of an agent-driven iterative loop.<n>The proposed method can be extended to deliver automated performance optimization for real production workloads.
  arXiv  Detail & Related papers  (2026-01-19T03:40:12Z)
• AccelOpt: A Self-Improving LLM Agentic System for AI Accelerator Kernel Optimization [12.269456144158783]
  We present AccelOpt, a self-improving large language model (LLM) agentic system that autonomously optimize kernels for emerging AI acclerators.<n>We build NKIBench, a new benchmark suite of AWS Trainium accelerator kernels with varying complexity extracted from real-world workloads.<n>Our evaluation confirms that AccelOpt's capability improves over time, boosting the average percentage of peak throughput from $49%$ to $61%$ on Trainium 1 and from $45%$ to $59%$ on NKIBench kernels.
  arXiv  Detail & Related papers  (2025-11-19T22:49:37Z)
• Eliminating Multi-GPU Performance Taxes: A Systems Approach to Efficient Distributed LLMs [61.953548065938385]
  We introduce the ''Three Taxes'' (Bulk Synchronous, Inter- Kernel Data Locality, and Kernel Launch Overhead) as an analytical framework.<n>We propose moving beyond the rigid BSP model to address key inefficiencies in distributed GPU execution.<n>We observe a 10-20% speedup in end-to-end latency over BSP-based approaches.
  arXiv  Detail & Related papers  (2025-11-04T01:15:44Z)
• INR-Arch: A Dataflow Architecture and Compiler for Arbitrary-Order Gradient Computations in Implicit Neural Representation Processing [66.00729477511219]
  Given a function represented as a computation graph, traditional architectures face challenges in efficiently computing its nth-order gradient. We introduce INR-Arch, a framework that transforms the computation graph of an nth-order gradient into a hardware-optimized dataflow architecture. We present results that demonstrate 1.8-4.8x and 1.5-3.6x speedup compared to CPU and GPU baselines respectively.
  arXiv  Detail & Related papers  (2023-08-11T04:24:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.