Model2Kernel: Model-Aware Symbolic Execution For Safe CUDA Kernels

• URL: http://arxiv.org/abs/2603.24595v1
• Date: Fri, 06 Mar 2026 04:13:28 GMT
• Title: Model2Kernel: Model-Aware Symbolic Execution For Safe CUDA Kernels
• Abstract summary: Production inference systems rely on kernels to implement core operations.<n>Memory-safety bugs can corrupt model weights, crash inference services, or even enable adversarial attacks.<n>This paper presents Model2 Kernel, the first practical system for automatically verifying memory safety of kernels used in inference.
• Score: 6.682299469816275
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: The widespread adoption of large language models (LLMs) has made GPU-accelerated inference a critical part of modern computing infrastructure. Production inference systems rely on CUDA kernels to implement core transformer operations, yet these kernels are highly susceptible to memory-safety bugs due to model-dependent tensor layouts, intricate memory indexing, and massive thread-level parallelism. Such bugs can corrupt model weights, crash inference services, or even enable adversarial attacks. Existing techniques either depend on unavailable hardware, incur high overhead, or fail to handle kernel inputs with variable lengths, and none can effectively detect CUDA memory bugs in LLM inference systems. This paper presents Model2Kernel, the first practical system for automatically verifying the memory safety of CUDA kernels used in LLM inference. Model2Kernel performs model-aware dynamic analysis to determine how each model invokes kernels and to classify kernel arguments as either fixed by the model architecture or controlled by model users. Using this information, Model2Kernel then applies CUDA-specialized symbolic execution, supported by new abstractions for dynamic tensor memory and thread identifiers, to accurately pinpoint memory bugs in kernels. In the evaluation on CUDA kernels and models from vLLM, Hugging Face, and recent LLM research papers, Model2Kernel discovers 353 previously unknown bugs while producing only nine false positives, demonstrating its effectiveness.

Related papers

• Enabling Memory-efficient Im2win Convolution with Multi-precision Support on GPU CUDA and Tensor Cores [26.80370423154318]
  Existing GPU convolution methods suffer from large memory overhead, poor cache utilization, limited effectiveness across kernel sizes, or numerical instability.<n>This work extends the im2win paradigm -- a universal, memory-efficient convolution method with contiguous memory access for all kernel sizes -- to run efficiently in full precision on cores and half precision on cores.
  arXiv  Detail & Related papers  (2026-08-21T04:09:32Z)
• Correct but Slow: An Empirical Study of the GPU Kernel Evaluation Gap in Modern Domain-Specific Languages [9.908756853186018]
  A functionally valid kernel may still fall far below the throughput of the optimized library operator it is intended to replace.<n>We study this correctness-performance gap using 22 Triton and TileLang kernels from five operator categories on NVIDIA A100 and GH200 GPUs.
  arXiv  Detail & Related papers  (2026-07-05T18:33:12Z)
• KernelBench-Verified: Do LLM-Generated Kernels Actually Beat PyTorch? [13.026983194416168]
  Large language models (LLMs) can generate custom kernels that appear to outperform PyTorch on benchmarks such as KernelBench.<n>We demonstrate that frontier models frequently engage in reward hacking to artificially inflate reported performance.
  arXiv  Detail & Related papers  (2026-06-26T03:22:50Z)
• GPU Forecasters: Language Models as Selective Surrogates for Kernel Runtime Optimization [73.92934811090163]
  We study how LLMs can serve as selective GPU surrogates for kernel evaluation.<n>We measure whether their forecasts are accurate, calibrated, and practically useful for recovering fast kernels under limited GPU-measurement budgets.<n>Our experiments demonstrate that LLMs can accurately forecast relative kernel performance, that their utility can be improved through reinforcement learning.
  arXiv  Detail & Related papers  (2026-05-29T15:56:08Z)
• FastKernels: Benchmarking GPU Kernel Generation in Production [15.284289018908781]
  LLM-based agents for GPU kernel generation are advancing rapidly.<n>Yet their progress is constrained by the benchmarks they optimize against.<n>Fast Kernels is a production-grade inference framework that runs at parity with hardened systems.
  arXiv  Detail & Related papers  (2026-05-22T04:19:04Z)
• CUDA Agent: Large-Scale Agentic RL for High-Performance CUDA Kernel Generation [51.72529978689561]
  Agent is a large-scale agentic reinforcement learning system that develops kernel expertise through three components.<n>Agent delivers 100%, 100%, and 92% faster rate over torchcompile on KernelBench.
  arXiv  Detail & Related papers  (2026-02-27T18:58:05Z)
• DICE: Diffusion Large Language Models Excel at Generating CUDA Kernels [17.979042914049842]
  Diffusion large language models (dLLMs) have emerged as a compelling alternative to autoregressive (AR) LLMs.<n>CuKe is an augmented dataset optimized for high-performance kernels.<n>DICE is a series of diffusion large language models designed for kernel generation.
  arXiv  Detail & Related papers  (2026-02-12T08:45:13Z)
• Outrunning LLM Cutoffs: A Live Kernel Crash Resolution Benchmark for All [57.23434868678603]
  Live-kBench is an evaluation framework for self-evolving benchmarks that scrapes and evaluates agents on freshly discovered kernel bugs.<n> kEnv is an agent-agnostic crash-resolution environment for kernel compilation, execution, and feedback.<n>Using kEnv, we benchmark three state-of-the-art agents, showing that they resolve 74% of crashes on the first attempt.
  arXiv  Detail & Related papers  (2026-02-02T19:06:15Z)
• dInfer: An Efficient Inference Framework for Diffusion Language Models [54.80918957287927]
  Diffusion-based large language models (dLLMs) have emerged as a promising alternative to autoregressive (AR) LLMs.<n>We present dInfer, an efficient and framework for dLLM inference.
  arXiv  Detail & Related papers  (2025-10-09T16:19:42Z)
• Joker: Joint Optimization Framework for Lightweight Kernel Machines [20.45405359815043]
  We propose Joker, a joint optimization framework for diverse kernel models, including KRR, logistic regression, and support vector machines.<n> Experiments show that Joker saves up to 90% memory but achieves comparable training time and performance (or even better) than the state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-05-23T11:36:45Z)
• vTensor: Flexible Virtual Tensor Management for Efficient LLM Serving [53.972175896814505]
  Large Language Models (LLMs) are widely used across various domains, processing millions of daily requests. Large Language Models (LLMs) are widely used across various domains, processing millions of daily requests.
  arXiv  Detail & Related papers  (2024-07-22T14:37:58Z)
• KGym: A Platform and Dataset to Benchmark Large Language Models on Linux Kernel Crash Resolution [59.20933707301566]
  Large Language Models (LLMs) are consistently improving at increasingly realistic software engineering (SE) tasks. In real-world software stacks, significant SE effort is spent developing foundational system software like the Linux kernel. To evaluate if ML models are useful while developing such large-scale systems-level software, we introduce kGym and kBench.
  arXiv  Detail & Related papers  (2024-07-02T21:44:22Z)
• Optimal Kernel Tuning Parameter Prediction using Deep Sequence Models [0.44998333629984877]
  We propose a methodology that uses deep sequence- to-sequence models to predict the optimal tuning parameters governing compute kernels. The proposed algorithm can achieve more than 90% accuracy on various convolutional kernels in MIOpen, the AMD machine learning primitives library.
  arXiv  Detail & Related papers  (2024-04-15T22:25:54Z)
• QUICK: Quantization-aware Interleaving and Conflict-free Kernel for efficient LLM inference [9.031180368026071]
  QUICK addresses the shared memory bank-conflict problem of state-of-the-art mixed precision matrix multiplication kernels. We demonstrate up to 1.91x speedup over existing kernels of AutoAWQ on larger batches and up to 1.94x throughput on representative LLM models on various NVIDIA GPU devices.
  arXiv  Detail & Related papers  (2024-02-15T16:38:41Z)
• Diagonal Memory Optimisation for Machine Learning on Micro-controllers [21.222568055417717]
  Micro controllers and low power CPUs are increasingly being used to perform inference with machine learning models. Small amounts of RAM available on these targets sets limits on size of models which can be executed. diagonal memory optimisation technique is described and shown to achieve memory savings of up to 34.5% when applied to eleven common models.
  arXiv  Detail & Related papers  (2020-10-04T19:45:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.