LUMINA: LLM-Guided GPU Architecture Exploration via Bottleneck Analysis

• URL: http://arxiv.org/abs/2603.05904v1
• Date: Fri, 06 Mar 2026 04:47:18 GMT
• Title: LUMINA: LLM-Guided GPU Architecture Exploration via Bottleneck Analysis
• Abstract summary: Large-Language Model (LLM) inference is challenging because of GPU's vast, multi-modal design spaces.<n>Existing automated DSE methods are often prohibitively expensive.<n>We present LUMINA, an LLM-driven GPU architecture exploration framework that leverage AI to enhance the DSE efficiency.
• Score: 8.471843667828994
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: GPU design space exploration (DSE) for modern AI workloads, such as Large-Language Model (LLM) inference, is challenging because of GPUs' vast, multi-modal design spaces, high simulation costs, and complex design optimization objectives (e.g. performance, power and area trade-offs). Existing automated DSE methods are often prohibitively expensive, either requiring an excessive number of exploration samples or depending on intricate, manually crafted analyses of interdependent critical paths guided by human heuristics. We present LUMINA, an LLM-driven GPU architecture exploration framework that leverage AI to enhance the DSE efficiency and efficacy for GPUs. LUMINA extracts architectural knowledge from simulator code and performs sensitivity studies to automatically compose DSE rules,which are auto-corrected during exploration. A core component of LUMINA is a DSE Benchmark that comprehensively evaluates and enhances LLMs' capabilities across three fundamental skills required for architecture optimization, which provides a principled and reproducible basis for model selection and ensuring consistent architectural reasoning. In the design space with 4.7 million possible samples, LUMINA identifies 6 designs of better performance and area than an A100 GPU efficiently, using only 20 steps via LLM-assisted bottleneck analysis. In comparison, LUMINA achieves 17.5x higher than design space exploration efficiency, and 32.9% better designs (i.e. Pareto Hypervolume) than Machine-Learning baselines, showcasing its ability to deliver high-quality design guidance with minimal search cost.

Related papers

• FrontierOR: Benchmarking LLMs' Capacity for Efficient Algorithm Design in Large-Scale Optimization [61.43300970020897]
  Large language models (LLMs) are increasingly used for optimization modeling and solver-code generation.<n>Existing benchmarks are limited to small or simplified examples far below real-world scale and complexity.<n>We introduce FrontierOR, among the first benchmarks to evaluate LLM-based efficient algorithm design for realistic large-scale optimization problems.
  arXiv  Detail & Related papers  (2026-05-24T20:10:42Z)
• Large Language Models for Physics Instrument Design [0.0]
  We study the use of large language models (LLMs) for physics instrument design and compare their performance to reinforcement learning (RL)<n>Although RL yields stronger final designs, we find that modern LLMs consistently generate valid, resource-aware, and physically meaningful configurations.<n>In doing so, they point toward automated, closed-loop instrument design in which much of the human effort required to structure and supervise optimization can be reduced.
  arXiv  Detail & Related papers  (2026-01-12T14:30:54Z)
• MM-HELIX: Boosting Multimodal Long-Chain Reflective Reasoning with Holistic Platform and Adaptive Hybrid Policy Optimization [103.74675519953898]
  Long-chain reflective reasoning is a prerequisite for solving complex real-world problems.<n>We build a benchmark consisting 1,260 samples of 42 challenging synthetic tasks.<n>We generate post-training data and explore learning paradigms for exploiting such data.
  arXiv  Detail & Related papers  (2025-10-09T17:53:58Z)
• MAHL: Multi-Agent LLM-Guided Hierarchical Chiplet Design with Adaptive Debugging [30.305211001929496]
  Large Language Models (LLMs) are promising to extend their abilities to 2.5D integration.<n>LLMs face challenges such as flatten design, high validation cost and imprecise parameter optimization.<n>We propose MAHL, a hierarchical LLM-based chiplet design generation framework.
  arXiv  Detail & Related papers  (2025-08-08T05:47:31Z)
• Teaching LLM to Reason: Reinforcement Learning from Algorithmic Problems without Code [76.80306464249217]
  We propose TeaR, which aims at teaching LLMs to reason better.<n>TeaR leverages careful data curation and reinforcement learning to guide models in discovering optimal reasoning paths through code-related tasks.<n>We conduct extensive experiments using two base models and three long-CoT distillation models, with model sizes ranging from 1.5 billion to 32 billion parameters, and across 17 benchmarks spanning Math, Knowledge, Code, and Logical Reasoning.
  arXiv  Detail & Related papers  (2025-07-10T07:34:05Z)
• GPU Kernel Scientist: An LLM-Driven Framework for Iterative Kernel Optimization [0.0]
  This paper introduces an automated methodology for iteratively refining accelerator kernels.<n>Our methodology employs LLMs in a multi-stage, evolutionary process.<n>We detail how this approach navigates the challenges of the AMD MI300 target architecture.
  arXiv  Detail & Related papers  (2025-06-25T19:59:34Z)
• Dynamic Acoustic Model Architecture Optimization in Training for ASR [51.21112094223223]
  DMAO is an architecture optimization framework that employs a grow-and-drop strategy to automatically reallocate parameters during training.<n>We evaluate DMAO through experiments with CTC onSpeech, TED-LIUM-v2 and Switchboard datasets.
  arXiv  Detail & Related papers  (2025-06-16T07:47:34Z)
• LLMs for Engineering: Teaching Models to Design High Powered Rockets [0.0]
  Large Language Models (LLMs) have transformed software engineering, but their application to physical engineering domains remains underexplored.<n>This paper evaluates LLMs' capabilities in high-powered rocketry design through RocketBench.<n>We test models on two increasingly complex design tasks: target altitude optimization and precision landing challenges.
  arXiv  Detail & Related papers  (2025-04-27T23:59:39Z)
• SEKI: Self-Evolution and Knowledge Inspiration based Neural Architecture Search via Large Language Models [11.670056503731905]
  We introduce SEKI, a novel large language model (LLM)-based neural architecture search (NAS) method.<n>Inspired by the chain-of-thought (CoT) paradigm in modern LLMs, SEKI operates in two key stages: self-evolution and knowledge distillation.
  arXiv  Detail & Related papers  (2025-02-27T09:17:49Z)
• EVOLvE: Evaluating and Optimizing LLMs For In-Context Exploration [76.66831821738927]
  Large language models (LLMs) remain under-studied in scenarios requiring optimal decision-making under uncertainty.<n>We measure LLMs' (in)ability to make optimal decisions in bandits, a state-less reinforcement learning setting relevant to many applications.<n>Motivated by the existence of optimal exploration algorithms, we propose efficient ways to integrate this algorithmic knowledge into LLMs.
  arXiv  Detail & Related papers  (2024-10-08T17:54:03Z)
• ArchGym: An Open-Source Gymnasium for Machine Learning Assisted Architecture Design [52.57999109204569]
  ArchGym is an open-source framework that connects diverse search algorithms to architecture simulators. We evaluate ArchGym across multiple vanilla and domain-specific search algorithms in designing custom memory controller, deep neural network accelerators, and custom SOC for AR/VR workloads.
  arXiv  Detail & Related papers  (2023-06-15T06:41:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.