HTAM: Hierarchical Transition-Attended Memory for Operator Optimization

• URL: http://arxiv.org/abs/2605.29734v1
• Date: Thu, 28 May 2026 10:29:01 GMT
• Title: HTAM: Hierarchical Transition-Attended Memory for Operator Optimization
• Abstract summary: Recent LLM-based code generation makes automatic operator generation promising, but operator optimization remains a hardware-aware search problem.<n>Existing LLM-based methods face a granularity mismatch: coarse hints are reusable but hard to execute, whereas detailed memories are actionable but enlarge the search space and obscure optimization bottlenecks.<n>This paper proposes HTAM, a coarse-to-fine framework for LLM-based operator optimization.
• Score: 32.20872995255167
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: High-performance GPU kernels are essential for efficient LLM deployment, yet optimizing them remains expertise-intensive. Recent LLM-based code generation makes automatic GPU operator generation promising, but operator optimization remains a hardware-aware search problem. Existing LLM-based methods face a granularity mismatch: coarse hints are reusable but hard to execute, whereas detailed memories are actionable but enlarge the search space and obscure optimization bottlenecks. The key challenge is therefore to organize optimization experience at an appropriate granularity. To address this issue, this paper proposes HTAM (Hierarchical Transition-Attended Memory), a coarse-to-fine framework for LLM-based operator optimization. HTAM builds a two-level Hierarchical Transition Graph (HTG) to organize coarse global directions, detailed local strategies, and transition experience between optimization steps. During each evolution step, HTAM selects a global direction from the current state and recent optimization history, retrieves the corresponding local strategy memory, and uses it to guide concrete CUDA code generation. Experiments on the full KernelBench suite demonstrate that HTAM consistently improves correctness, fast-solution rate, and speedup over LLM-based baselines, while backend and Robust-KBench studies indicate transferable benefits from structured memory.

Related papers

• Effect of Abstractions and Prompting Strategies on LLM-Guided High-Performance Optimizations [0.0]
  Large Language Models (LLMs) are capable of generating and understanding code.<n>Traditionally, this automation involves transcribing the source code into a domain-specific representation.<n>This paper investigates whether the traditional abstractions used in automated code optimization improve the performance and correctness of LLM-guided optimizations.
  arXiv  Detail & Related papers  (2026-08-08T12:12:28Z)
• AUTOGATE: Automated Clock Gating via Toggling-Aware LLM-based RTL Rewriting [17.55906131128768]
  We present AUTOGATE, the first agentic framework for industry-grade RTL power optimization.<n>We introduce a Machine Learning (ML)-LLM co-design that bridges waveform-level analysis and RTL rewriting.<n>We show that AUTOGATE consistently reduces dynamic power relative to baselines.
  arXiv  Detail & Related papers  (2026-06-16T03:22:54Z)
• BEAM: Bi-level Memory-adaptive Algorithmic Evolution for LLM-Powered Heuristic Design [49.50107918295052]
  Large Language Model-based Hyper Heuristic (LHH) has recently emerged as an efficient way for automatic design.<n>We propose textbfBEAM (Bi-level Memory-adaptive Algorithmic Evolution) to address this problem.<n>We show that BEAM significantly outperforms existing LHHs, notably reducing the optimality gap by 37.84% on aggregate.
  arXiv  Detail & Related papers  (2026-04-14T15:46:47Z)
• VeriAgent: A Tool-Integrated Multi-Agent System with Evolving Memory for PPA-Aware RTL Code Generation [52.43796056267479]
  We propose a PPA-aware, tool-integrated multi-agent framework for high-quality verilog code generation.<n>Our framework explicitly incorporates EDA tools into a closed-loop workflow composed of a textitProgrammer Agent, a textitCorrectness Agent, and a textitPPA Agent<n>To support continuous improvement without model retraining, we introduce an textitEvolved Memory Mechanism that externalizes optimization experience into structured memory nodes.
  arXiv  Detail & Related papers  (2026-03-18T11:25:40Z)
• Agentic Memory: Learning Unified Long-Term and Short-Term Memory Management for Large Language Model Agents [57.38404718635204]
  Large language model (LLM) agents face fundamental limitations in long-horizon reasoning due to finite context windows.<n>Existing methods typically handle long-term memory (LTM) and short-term memory (STM) as separate components.<n>We propose Agentic Memory (AgeMem), a unified framework that integrates LTM and STM management directly into the agent's policy.
  arXiv  Detail & Related papers  (2026-01-05T08:24:16Z)
• QiMeng-Kernel: Macro-Thinking Micro-Coding Paradigm for LLM-Based High-Performance GPU Kernel Generation [41.53673797546332]
  Micro Coding is a hierarchical framework inspired by the staged optimization strategy of human experts.<n>It decouples optimization strategy from implementation details, ensuring efficiency through high-level strategy and correctness through low-level implementation.<n>It achieves near 100% and 70% accuracy at Levels 1-2 and 3, over 50% than SOTA general-purpose and domain-finetuned LLMs, with up to 7.3x speedup over LLMs, and 2.2x over expert-optimized PyTorch Eager kernels.
  arXiv  Detail & Related papers  (2025-11-25T09:17:47Z)
• STARK: Strategic Team of Agents for Refining Kernels [23.717055490630596]
  We introduce an agentic framework for GPU kernel optimization that explores the design space through multi-agent collaboration.<n>This framework mimics the workflow of expert engineers, enabling LLMs to reason about hardware trade-offs, incorporate profiling feedback, and refine kernels iteratively.<n>We evaluate our approach on KernelBench, a benchmark for LLM-based kernel optimization, and demonstrate substantial improvements over baseline agents.
  arXiv  Detail & Related papers  (2025-10-19T20:41:46Z)
• COSMOS: A Hybrid Adaptive Optimizer for Memory-Efficient Training of LLMs [77.79640601822341]
  Large Language Models (LLMs) have demonstrated remarkable success across various domains.<n>Their optimization remains a significant challenge due to the complex and high-dimensional loss landscapes they inhabit.
  arXiv  Detail & Related papers  (2025-02-24T18:42:19Z)
• LLM as a Complementary Optimizer to Gradient Descent: A Case Study in Prompt Tuning [69.95292905263393]
  We show that gradient-based and high-level LLMs can effectively collaborate a combined optimization framework.<n>In this paper, we show that these complementary to each other and can effectively collaborate a combined optimization framework.
  arXiv  Detail & Related papers  (2024-05-30T06:24:14Z)
• Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmark [166.40879020706151]
  This paper proposes a shift towards BP-free, zeroth-order (ZO) optimization as a solution for reducing memory costs during fine-tuning. Unlike traditional ZO-SGD methods, our work expands the exploration to a wider array of ZO optimization techniques. Our study unveils previously overlooked optimization principles, highlighting the importance of task alignment, the role of the forward gradient method, and the balance between algorithm complexity and fine-tuning performance.
  arXiv  Detail & Related papers  (2024-02-18T14:08:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.