PEL-NAS: Search Space Partitioned Architecture Prompt Co-Evolutionary LLM-driven Hardware-Aware Neural Architecture Search

• URL: http://arxiv.org/abs/2510.01472v2
• Date: Sat, 04 Oct 2025 17:40:31 GMT
• Title: PEL-NAS: Search Space Partitioned Architecture Prompt Co-Evolutionary LLM-driven Hardware-Aware Neural Architecture Search
• Authors: Hengyi Zhu, Grace Li Zhang, Shaoyi Huang,
• Abstract summary: Hardware-Aware Neural Architecture Search (HW-NAS) requires joint optimization of accuracy and latency under device constraints.<n>We propose PEL-NAS: a search space Partitioned, architecture prompt co-Evolutionary and LLM-driven Neural Architecture Search.
• Score: 3.59587275290785
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Hardware-Aware Neural Architecture Search (HW-NAS) requires joint optimization of accuracy and latency under device constraints. Traditional supernet-based methods require multiple GPU days per dataset. Large Language Model (LLM)-driven approaches avoid training a large supernet and can provide quick feedback, but we observe an exploration bias: the LLM repeatedly proposes neural network designs within limited search space and fails to discover architectures across different latency ranges in the entire search space. To address this issue, we propose PEL-NAS: a search space Partitioned, architecture prompt co-Evolutionary and LLM-driven Neural Architecture Search that can generate neural networks with high accuracy and low latency with reduced search cost. Our proposed PEL-NAS has three key components: 1) a complexity-driven partitioning engine that divides the search space by complexity to enforce diversity and mitigate exploration bias; 2) an LLM-powered architecture prompt co-evolution operator, in which the LLM first updates a knowledge base of design heuristics based on results from the previous round, then performs a guided evolution algorithm on architectures with prompts that incorporate this knowledge base. Prompts and designs improve together across rounds which avoids random guesswork and improve efficiency; 3) a zero-cost predictor to avoid training a large number of candidates from scratch. Experimental results show that on HW-NAS-Bench, PEL-NAS can achieve overall higher HV, lower IGD, and up to 54% lower latency than baselines at similar accuracy. Meanwhile, the search cost drops from days to minutes compared with traditional supernet baselines.

Related papers

• Delta-NAS: Difference of Architecture Encoding for Predictor-based Evolutionary Neural Architecture Search [5.1331676121360985]
  We craft an algorithm with the capability to perform fine-grain NAS at a low cost. We propose projecting the problem to a lower dimensional space through predicting the difference in accuracy of a pair of similar networks.
  arXiv  Detail & Related papers  (2024-11-21T02:43:32Z)
• A Pairwise Comparison Relation-assisted Multi-objective Evolutionary Neural Architecture Search Method with Multi-population Mechanism [56.09418231453024]
  Neural architecture search (NAS) enables researchers to automatically explore vast search spaces and find efficient neural networks.<n>NAS suffers from a key bottleneck, i.e., numerous architectures need to be evaluated during the search process.<n>We propose the SMEM-NAS, a pairwise comparison relation-assisted multi-objective evolutionary algorithm based on a multi-population mechanism.
  arXiv  Detail & Related papers  (2024-07-22T12:46:22Z)
• Towards Less Constrained Macro-Neural Architecture Search [2.685668802278155]
  Neural Architecture Search (NAS) networks achieve state-of-the-art performance in a variety of tasks. Most NAS methods rely heavily on human-defined assumptions that constrain the search. We present experiments showing that LCMNAS generates state-of-the-art architectures from scratch with minimal GPU computation.
  arXiv  Detail & Related papers  (2022-03-10T17:53:03Z)
• $\eta$-DARTS: Beta-Decay Regularization for Differentiable Architecture Search [85.84110365657455]
  We propose a simple-but-efficient regularization method, termed as Beta-Decay, to regularize the DARTS-based NAS searching process. Experimental results on NAS-Bench-201 show that our proposed method can help to stabilize the searching process and makes the searched network more transferable across different datasets.
  arXiv  Detail & Related papers  (2022-03-03T11:47:14Z)
• L$^{2}$NAS: Learning to Optimize Neural Architectures via Continuous-Action Reinforcement Learning [23.25155249879658]
  Differentiable architecture search (NAS) achieved remarkable results in deep neural network design. We show that L$2$ achieves state-of-theart results on DART201 benchmark as well as NASS and Once-for-All search policies.
  arXiv  Detail & Related papers  (2021-09-25T19:26:30Z)
• FLASH: Fast Neural Architecture Search with Hardware Optimization [7.263481020106725]
  Neural architecture search (NAS) is a promising technique to design efficient and high-performance deep neural networks (DNNs) This paper proposes FLASH, a very fast NAS methodology that co-optimizes the DNN accuracy and performance on a real hardware platform.
  arXiv  Detail & Related papers  (2021-08-01T23:46:48Z)
• FNAS: Uncertainty-Aware Fast Neural Architecture Search [54.49650267859032]
  Reinforcement learning (RL)-based neural architecture search (NAS) generally guarantees better convergence yet suffers from the requirement of huge computational resources. We propose a general pipeline to accelerate the convergence of the rollout process as well as the RL process in NAS. Experiments on the Mobile Neural Architecture Search (MNAS) search space show the proposed Fast Neural Architecture Search (FNAS) accelerates standard RL-based NAS process by 10x.
  arXiv  Detail & Related papers  (2021-05-25T06:32:52Z)
• BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Search [100.28980854978768]
  We present Block-wisely Self-supervised Neural Architecture Search (BossNAS) We factorize the search space into blocks and utilize a novel self-supervised training scheme, named ensemble bootstrapping, to train each block separately. We also present HyTra search space, a fabric-like hybrid CNN-transformer search space with searchable down-sampling positions.
  arXiv  Detail & Related papers  (2021-03-23T10:05:58Z)
• ISTA-NAS: Efficient and Consistent Neural Architecture Search by Sparse Coding [86.40042104698792]
  We formulate neural architecture search as a sparse coding problem. In experiments, our two-stage method on CIFAR-10 requires only 0.05 GPU-day for search. Our one-stage method produces state-of-the-art performances on both CIFAR-10 and ImageNet at the cost of only evaluation time.
  arXiv  Detail & Related papers  (2020-10-13T04:34:24Z)
• MS-RANAS: Multi-Scale Resource-Aware Neural Architecture Search [94.80212602202518]
  We propose Multi-Scale Resource-Aware Neural Architecture Search (MS-RANAS) We employ a one-shot architecture search approach in order to obtain a reduced search cost. We achieve state-of-the-art results in terms of accuracy-speed trade-off.
  arXiv  Detail & Related papers  (2020-09-29T11:56:01Z)
• ADWPNAS: Architecture-Driven Weight Prediction for Neural Architecture Search [6.458169480971417]
  We propose an Architecture-Driven Weight Prediction (ADWP) approach for neural architecture search (NAS) In our approach, we first design an architecture-intensive search space and then train a HyperNetwork by inputting encoding architecture parameters. Results show that one search procedure can be completed in 4.0 GPU hours on CIFAR-10.
  arXiv  Detail & Related papers  (2020-03-03T05:06:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.