CuBridge: An LLM-Based Framework for Understanding and Reconstructing High-Performance Attention Kernels

• URL: http://arxiv.org/abs/2605.05023v1
• Date: Wed, 06 May 2026 15:19:07 GMT
• Title: CuBridge: An LLM-Based Framework for Understanding and Reconstructing High-Performance Attention Kernels
• Abstract summary: CuBridge is a framework that adapts expert-written attention kernels through a structured lift-transfer-lower workflow.<n>CuBridge consistently produces correct kernels and substantially outperforms general frameworks, compiler-based approaches, and prior LLM-based methods.
• Score: 36.66718851543183
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Efficient CUDA implementations of attention mechanisms are critical to modern deep learning systems, yet supporting diverse and evolving attention variants remains challenging. Existing frameworks and compilers trade performance for flexibility, while expert-written kernels achieve high efficiency but are difficult to adapt. Recent work explores large language models (LLMs) for GPU kernel generation, but prior studies report unstable correctness and significant performance gaps for complex operators such as attention. We present CuBridge, an LLM-based framework that adapts expert-written attention kernels through a structured lift-transfer-lower workflow. CuBridge starts from expert-written CUDA attention kernels and lifts them into an executable intermediate representation that makes execution orchestration explicit while abstracting low-level CUDA syntax. Given a user-provided PyTorch specification, CuBridge generates and verifies a target IR program, then reconstructs optimized CUDA code via reference-guided lowering. Across diverse attention variants and GPU platforms, CuBridge consistently produces correct kernels and substantially outperforms general frameworks, compiler-based approaches, and prior LLM-based methods.

Related papers

• CUDA-Harness: Harnessing Agentic CUDA Kernel Generation and Optimization from Natural Language [12.536212788514815]
  Text2CUDA makes generating kernels directly from natural language (Text2CUDA) essential.<n> Large Language Models (LLMs) prompts a series of works exploring LLM-based kernel generation.<n>We propose a framework for harnessing agentic kernel-level optimization from natural language.
  arXiv  Detail & Related papers  (2026-08-30T13:51:43Z)
• CUDAHercules: Benchmarking Hardware-Aware Expert-level CUDA Optimization for LLMs [45.52512820646709]
  We introduce Hercules, a benchmark that evaluates generated against end-to-end human-expert SOTA systems.<n>We show that automated programming remains far from fully solved and requires stronger hardware reasoning, better tool use, and training objectives.
  arXiv  Detail & Related papers  (2026-05-08T20:35:27Z)
• CuTeGen: An LLM-Based Agentic Framework for Generation and Optimization of High-Performance GPU Kernels using CuTe [7.881930340027473]
  CuTeGen is a framework for automated generation and optimization of GPU kernels.<n>It treats kernel development as a structured generate--test--refine workflow.<n>We show that CuTeGen produces functionally correct kernels and achieves competitive performance relative to optimized library implementations.
  arXiv  Detail & Related papers  (2026-04-01T23:55:23Z)
• CUCo: An Agentic Framework for Compute and Communication Co-design [13.906670325427463]
  CUCo is a training-free agent-driven workflow that automatically generates high-performance kernels that jointly orchestrate and communication.<n>By co-optimizing these traditionally disjoint components, CUCo unlocks new optimization opportunities unavailable to existing approaches.
  arXiv  Detail & Related papers  (2026-03-02T20:35:50Z)
• CUDA Agent: Large-Scale Agentic RL for High-Performance CUDA Kernel Generation [51.72529978689561]
  Agent is a large-scale agentic reinforcement learning system that develops kernel expertise through three components.<n>Agent delivers 100%, 100%, and 92% faster rate over torchcompile on KernelBench.
  arXiv  Detail & Related papers  (2026-02-27T18:58:05Z)
• KernelBlaster: Continual Cross-Task CUDA Optimization via Memory-Augmented In-Context Reinforcement Learning [3.4998382481249286]
  We release KernelBlaster as an open-source agentic framework, accompanied by a test harness, verification components, and a reproducible evaluation.<n>Our method achieves mean speed-ups of 1.43x, 2.50x, and 1.50x on KernelBench Levels 1, 2, and 3, respectively.
  arXiv  Detail & Related papers  (2026-02-15T19:48:43Z)
• QiMeng-NeuComBack: Self-Evolving Translation from IR to Assembly Code [52.66657751895655]
  Large Language Models (LLMs) offer a compelling new paradigm: Neural Compilation.<n>This paper introduces NeuComBack, a novel benchmark dataset specifically designed for IR-to-assembly compilation.<n>We propose a self-evolving prompt optimization method that enables LLMs to evolve their internal prompt strategies.
  arXiv  Detail & Related papers  (2025-11-03T03:20:26Z)
• Towards Robust Agentic CUDA Kernel Benchmarking, Verification, and Optimization [25.135006275638172]
  We introduce robust-kbench, a new benchmark for rigorous evaluation of kernel performance and correctness across varied scenarios.<n>We also present a comprehensive agentic framework that translates torch code to kernels and iteratively improve their runtime setting.<n>Our approach produces kernels outperforming torch implementations for practical applications, including forward and backward passes.
  arXiv  Detail & Related papers  (2025-09-16T11:08:30Z)
• Harnessing Deep Learning and HPC Kernels via High-Level Loop and Tensor Abstractions on CPU Architectures [67.47328776279204]
  This work introduces a framework to develop efficient, portable Deep Learning and High Performance Computing kernels. We decompose the kernel development in two steps: 1) Expressing the computational core using Processing Primitives (TPPs) and 2) Expressing the logical loops around TPPs in a high-level, declarative fashion. We demonstrate the efficacy of our approach using standalone kernels and end-to-end workloads that outperform state-of-the-art implementations on diverse CPU platforms.
  arXiv  Detail & Related papers  (2023-04-25T05:04:44Z)
• Extending C++ for Heterogeneous Quantum-Classical Computing [56.782064931823015]
  qcor is a language extension to C++ and compiler implementation that enables heterogeneous quantum-classical programming, compilation, and execution in a single-source context. Our work provides a first-of-its-kind C++ compiler enabling high-level quantum kernel (function) expression in a quantum-language manner.
  arXiv  Detail & Related papers  (2020-10-08T12:49:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.