ZAPS: Zero-Cost Active Proxy Search for Neural Architecture Search

• URL: http://arxiv.org/abs/2609.14184v1
• Date: Sat, 12 Sep 2026 23:34:41 GMT
• Title: ZAPS: Zero-Cost Active Proxy Search for Neural Architecture Search
• Abstract summary: We introduce ZAPS (Zero-cost Active Proxy Search), a four-stage pipeline that closes this gap.<n>ZAPS selects a compact, non-redundant proxy subset offline via ProxyFit; (ii) seeds the search with a hybrid K-means strategy that balances exploitation and exploration; (iii) re-selects proxies at every iteration by a boot vote as the labeled set grows; and (iv) ranks candidates with an XGBoost ensemble trained jointly on proxy ranks and one-hot topological encodings.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Neural Architecture Search (NAS) automates network design, but evaluating a single candidate requires training it to convergence, making exhaustive search intractable. Zero-cost proxies estimate architecture quality at initialization in seconds, yet a single proxy is noisy, and combining several does not straightforwardly help: proxies are strongly correlated, so naive aggregation compounds their shared errors instead of averaging them out. Existing methods exploit either proxy signals or architectural topology - never both within a single active-learning framework. We introduce ZAPS (Zero-cost Active Proxy Search), a four-stage pipeline that closes this gap. ZAPS (i) selects a compact, non-redundant proxy subset offline via ProxyFit, a greedy anti-redundancy criterion; (ii) seeds the search with a hybrid K-means strategy that balances exploitation and exploration; (iii) re-selects proxies at every iteration by a bootstrapped vote as the labeled set grows; and (iv) ranks candidates with an XGBoost ensemble trained jointly on proxy ranks and one-hot topological encodings, queried through an Upper Confidence Bound (UCB) acquisition function. On NAS-Bench-201 under a budget of B=200 evaluations, ZAPS recovers 52.3% of the true top-100 architectures on CIFAR-10 and 65.8% on CIFAR-100, ahead of every baseline we consider - Random Search, Local Search, REA, BANANAS and TPE - and, on CIFAR-10, with less than half the run-to-run standard deviation of the strongest of them. The advantage is largest where evaluations are scarce: on NAS-Bench-201 it narrows as the budget grows, whereas on the harder NAS-Bench-101, which no method comes close to saturating, it widens instead. All methods are scored by a single criterion: how much of the true top-100 lies among the architectures they actually evaluated.

Related papers

• CoRA-NAS: Coarse Ranking and Anchor-Residual Refinement for Neural Architecture Search [9.443798158626539]
  CoRA-NAS is a two-stage framework combining a static ranking prior with low-cost learning-curve refinement.<n>CoRA-Rank aggregates capacity and structure-at-initialization proxies through an equal-weight log-rank consensus.<n>CoRA-Refine samples anchors across this prior, extrapolates their early validation curves, and propagates a learned residual correction with an ExtraTrees model.
  arXiv  Detail & Related papers  (2026-09-10T17:49:19Z)
• Bi-EZP: LLM-Guided Bilevel Program Evolution for Ensemble Zero-Cost Proxy Discovery [2.751603258501323]
  Bi-EZP is a bilevel framework that separates program discovery from numerical calibration.<n>Results show that separating program discovery from numerical calibration provides an effective approach to automated ensemble zero-cost proxy construction.
  arXiv  Detail & Related papers  (2026-08-22T11:34:35Z)
• Claudini: Autoresearch Discovers State-of-the-Art Adversarial Attack Algorithms for LLMs [75.62489398666644]
  We show that an emphautoresearch-style pipeline citepkarpathy2026autoresearch powered by Claude Code discovers novel white-box adversarial attack.<n>textbfsignificantly outperform all existing (30+ methods) in jailbreaking and prompt injection evaluations.
  arXiv  Detail & Related papers  (2026-03-25T16:50:56Z)
• Zero-Shot Neural Architecture Search with Weighted Response Correlation [5.953576590699768]
  We present a novel training-free estimation proxy called weighted response correlation (WRCor)<n>WRCor utilizes coefficient correlation matrices of responses across different input samples to calculate the proxy scores of estimated architectures.<n>Our NAS algorithm can discover an architecture with a 22.1% test error on the ImageNet-1k dataset within 4 GPU hours.
  arXiv  Detail & Related papers  (2025-07-08T02:19:29Z)
• $\beta$-DARTS++: Bi-level Regularization for Proxy-robust Differentiable Architecture Search [96.99525100285084]
  Regularization method, Beta-Decay, is proposed to regularize the DARTS-based NAS searching process (i.e., $beta$-DARTS) In-depth theoretical analyses on how it works and why it works are provided.
  arXiv  Detail & Related papers  (2023-01-16T12:30:32Z)
• Extensible Proxy for Efficient NAS [38.124755703499886]
  We propose a new approach to design deep neural networks (DNNs) called Neural Architecture Search (NAS) NAS proxies are proposed to address the demanding computational issues of NAS, where each candidate architecture network only requires one iteration of backpropagation. Our experiments confirm the effectiveness of both Eproxy and Eproxy+DPS.
  arXiv  Detail & Related papers  (2022-10-17T22:18:22Z)
• Generalizing Few-Shot NAS with Gradient Matching [165.5690495295074]
  One-Shot methods train one supernet to approximate the performance of every architecture in the search space via weight-sharing. Few-Shot NAS reduces the level of weight-sharing by splitting the One-Shot supernet into multiple separated sub-supernets. It significantly outperforms its Few-Shot counterparts while surpassing previous comparable methods in terms of the accuracy of derived architectures.
  arXiv  Detail & Related papers  (2022-03-29T03:06:16Z)
• Non-isotropy Regularization for Proxy-based Deep Metric Learning [78.18860829585182]
  We propose non-isotropy regularization ($mathbbNIR$) for proxy-based Deep Metric Learning. This allows us to explicitly induce a non-isotropic distribution of samples around a proxy to optimize for. Experiments highlight consistent generalization benefits of $mathbbNIR$ while achieving competitive and state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-03-16T11:13:20Z)
• ProxyBO: Accelerating Neural Architecture Search via Bayesian Optimization with Zero-cost Proxies [30.059154132130207]
  We present ProxyBO, an efficient Bayesian optimization framework that utilizes zero-cost proxies to accelerate neural architecture search. We show that ProxyBO consistently outperforms competitive baselines on five tasks from three public benchmarks.
  arXiv  Detail & Related papers  (2021-10-20T08:18:16Z)
• Fewer is More: A Deep Graph Metric Learning Perspective Using Fewer Proxies [65.92826041406802]
  We propose a Proxy-based deep Graph Metric Learning approach from the perspective of graph classification. Multiple global proxies are leveraged to collectively approximate the original data points for each class. We design a novel reverse label propagation algorithm, by which the neighbor relationships are adjusted according to ground-truth labels.
  arXiv  Detail & Related papers  (2020-10-26T14:52:42Z)
• DC-NAS: Divide-and-Conquer Neural Architecture Search [108.57785531758076]
  We present a divide-and-conquer (DC) approach to effectively and efficiently search deep neural architectures. We achieve a $75.1%$ top-1 accuracy on the ImageNet dataset, which is higher than that of state-of-the-art methods using the same search space.
  arXiv  Detail & Related papers  (2020-05-29T09:02:16Z)
• EcoNAS: Finding Proxies for Economical Neural Architecture Search [130.59673917196994]
  In this paper, we observe that most existing proxies exhibit different behaviors in maintaining the rank consistency among network candidates. Inspired by these observations, we present a reliable proxy and further formulate a hierarchical proxy strategy. The strategy spends more computations on candidate networks that are potentially more accurate, while discards unpromising ones in early stage with a fast proxy.
  arXiv  Detail & Related papers  (2020-01-05T13:29:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.