SEval-NAS: A Search-Agnostic Evaluation for Neural Architecture Search

• URL: http://arxiv.org/abs/2603.00099v1
• Date: Tue, 17 Feb 2026 15:02:02 GMT
• Title: SEval-NAS: A Search-Agnostic Evaluation for Neural Architecture Search
• Abstract summary: We propose SEval-NAS, a metric-evaluation mechanism that converts architectures to strings, embeds them as vectors, and predicts performance metrics.<n>Using NATS-Bench and HW-NAS-Bench, we evaluated accuracy, latency, and memory.<n>We further integrated SEval-NAS into FreeREA to evaluate metrics not originally included.
• Score: 20.37165647595783
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Neural architecture search (NAS) automates the discovery of neural networks that meet specified criteria, yet its evaluation procedures are often hardcoded, limiting the ability to introduce new metrics. This issue is especially pronounced in hardware-aware NAS, where objectives depend on target devices such as edge hardware. To address this limitation, we propose SEval-NAS, a metric-evaluation mechanism that converts architectures to strings, embeds them as vectors, and predicts performance metrics. Using NATS-Bench and HW-NAS-Bench, we evaluated accuracy, latency, and memory. Kendall's $τ$ correlations showed stronger latency and memory predictions than accuracy, indicating the suitability of SEval-NAS as a hardware cost predictor. We further integrated SEval-NAS into FreeREA to evaluate metrics not originally included. The method successfully ranked FreeREA-generated architectures, maintained search time, and required minimal algorithmic changes. Our implementation is available at: https://github.com/Analytics-Everywhere-Lab/neural-architecture-search

Related papers

• A Pairwise Comparison Relation-assisted Multi-objective Evolutionary Neural Architecture Search Method with Multi-population Mechanism [56.09418231453024]
  Neural architecture search (NAS) enables researchers to automatically explore vast search spaces and find efficient neural networks.<n>NAS suffers from a key bottleneck, i.e., numerous architectures need to be evaluated during the search process.<n>We propose the SMEM-NAS, a pairwise comparison relation-assisted multi-objective evolutionary algorithm based on a multi-population mechanism.
  arXiv  Detail & Related papers  (2024-07-22T12:46:22Z)
• DNA Family: Boosting Weight-Sharing NAS with Block-Wise Supervisions [121.05720140641189]
  We develop a family of models with the distilling neural architecture (DNA) techniques. Our proposed DNA models can rate all architecture candidates, as opposed to previous works that can only access a sub- search space using algorithms. Our models achieve state-of-the-art top-1 accuracy of 78.9% and 83.6% on ImageNet for a mobile convolutional network and a small vision transformer, respectively.
  arXiv  Detail & Related papers  (2024-03-02T22:16:47Z)
• MicroNAS: Memory and Latency Constrained Hardware-Aware Neural Architecture Search for Time Series Classification on Microcontrollers [3.0723404270319685]
  We adapt the concept of differentiable neural architecture search (DNAS) to solve the time-series classification problem on resource-constrained microcontrollers (MCUs) We introduce MicroNAS, a domain-specific HW-NAS system integration of DNAS, Lookup Tables, dynamic convolutions and a novel search space specifically designed for time-series classification on MCUs. Our studies on different MCUs and standard benchmark datasets demonstrate that MicroNAS finds MCU-tailored architectures that achieve performance (F1-score) near to state-of-the-art desktop models.
  arXiv  Detail & Related papers  (2023-10-27T06:55:15Z)
• Search-time Efficient Device Constraints-Aware Neural Architecture Search [6.527454079441765]
  Deep learning techniques like computer vision and natural language processing can be computationally expensive and memory-intensive. We automate the construction of task-specific deep learning architectures optimized for device constraints through Neural Architecture Search (NAS) We present DCA-NAS, a principled method of fast neural network architecture search that incorporates edge-device constraints.
  arXiv  Detail & Related papers  (2023-07-10T09:52:28Z)
• GeNAS: Neural Architecture Search with Better Generalization [14.92869716323226]
  Recent neural architecture search (NAS) approaches rely on validation loss or accuracy to find the superior network for the target data. In this paper, we investigate a new neural architecture search measure for excavating architectures with better generalization.
  arXiv  Detail & Related papers  (2023-05-15T12:44:54Z)
• Data Aware Neural Architecture Search [0.12891210250935145]
  In Machine Learning, one single metric is not enough to evaluate a NN architecture. Recent works on NAS for resource constrained systems have investigated various approaches to optimize for multiple metrics. We name such a system "Data Aware NAS", and we provide experimental evidence of its benefits.
  arXiv  Detail & Related papers  (2023-04-04T14:20:36Z)
• Neural Architecture Search: Two Constant Shared Weights Initialisations [0.0]
  epsinas is a novel zero-cost NAS metric that assesses architecture potential using two constant shared weight initialisations and the statistics of their outputs.<n>We show that the dispersion of raw outputs, normalised by their average magnitude, strongly correlates with trained accuracy.<n>Our computation requires no data labels, operates on a single minibatch, and eliminates the need for gradient.
  arXiv  Detail & Related papers  (2023-02-09T02:25:38Z)
• Generalization Properties of NAS under Activation and Skip Connection Search [66.8386847112332]
  We study the generalization properties of Neural Architecture Search (NAS) under a unifying framework. We derive the lower (and upper) bounds of the minimum eigenvalue of the Neural Tangent Kernel (NTK) under the (in)finite-width regime. We show how the derived results can guide NAS to select the top-performing architectures, even in the case without training.
  arXiv  Detail & Related papers  (2022-09-15T12:11:41Z)
• NAS-FCOS: Efficient Search for Object Detection Architectures [113.47766862146389]
  We propose an efficient method to obtain better object detectors by searching for the feature pyramid network (FPN) and the prediction head of a simple anchor-free object detector. With carefully designed search space, search algorithms, and strategies for evaluating network quality, we are able to find top-performing detection architectures within 4 days using 8 V100 GPUs.
  arXiv  Detail & Related papers  (2021-10-24T12:20:04Z)
• Weak NAS Predictors Are All You Need [91.11570424233709]
  Recent predictor-based NAS approaches attempt to solve the problem with two key steps: sampling some architecture-performance pairs and fitting a proxy accuracy predictor. We shift the paradigm from finding a complicated predictor that covers the whole architecture space to a set of weaker predictors that progressively move towards the high-performance sub-space. Our method costs fewer samples to find the top-performance architectures on NAS-Bench-101 and NAS-Bench-201, and it achieves the state-of-the-art ImageNet performance on the NASNet search space.
  arXiv  Detail & Related papers  (2021-02-21T01:58:43Z)
• MS-RANAS: Multi-Scale Resource-Aware Neural Architecture Search [94.80212602202518]
  We propose Multi-Scale Resource-Aware Neural Architecture Search (MS-RANAS) We employ a one-shot architecture search approach in order to obtain a reduced search cost. We achieve state-of-the-art results in terms of accuracy-speed trade-off.
  arXiv  Detail & Related papers  (2020-09-29T11:56:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.