CoRA-NAS: Coarse Ranking and Anchor-Residual Refinement for Neural Architecture Search

• URL: http://arxiv.org/abs/2609.11884v1
• Date: Thu, 10 Sep 2026 17:49:19 GMT
• Title: CoRA-NAS: Coarse Ranking and Anchor-Residual Refinement for Neural Architecture Search
• Abstract summary: CoRA-NAS is a two-stage framework combining a static ranking prior with low-cost learning-curve refinement.<n>CoRA-Rank aggregates capacity and structure-at-initialization proxies through an equal-weight log-rank consensus.<n>CoRA-Refine samples anchors across this prior, extrapolates their early validation curves, and propagates a learned residual correction with an ExtraTrees model.
• Score: 9.443798158626539
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Zero-cost proxies rank architectures cheaply, but their reliability varies across search spaces. We introduce CoRA-NAS (COarse Ranking + Anchor-residual), a two-stage framework combining a static ranking prior with low-cost learning-curve refinement. CoRA-Rank aggregates capacity and structure-at-initialization proxies through an equal-weight log-rank consensus and a target-free consensus gate. CoRA-Refine samples anchors across this prior, extrapolates their early validation curves, and propagates a learned residual correction with an ExtraTrees model. The refinement uses approximately 1% of the cost of fully training the candidate set. Fully trained architecture-accuracy labels are not used to fit the ranker. One configuration is used across spaces, with space-specific architecture encodings. Across NAS-Bench-201, NAS-Bench-101, TransNAS-Bench-101, and NATS-SSS, CoRA-Refine achieves mean Spearman correlations of 0.946, 0.715, 0.786, and 0.894, respectively. Its worst-space correlation of 0.715 is the highest among the compared methods. On NAS-Bench-201/CIFAR-100, its selected architecture reaches 73.32% accuracy, near the reported ground-truth best of 73.37%. On the pure size space, refinement recovers the static prior's shortfall relative to parameter count, while remaining tied with the strongest capacity proxies within noise. The resulting framework combines cross-space ranking robustness with low-cost architecture selection.

Related papers

• ZAPS: Zero-Cost Active Proxy Search for Neural Architecture Search [0.0]
  We introduce ZAPS (Zero-cost Active Proxy Search), a four-stage pipeline that closes this gap.<n>ZAPS selects a compact, non-redundant proxy subset offline via ProxyFit; (ii) seeds the search with a hybrid K-means strategy that balances exploitation and exploration; (iii) re-selects proxies at every iteration by a boot vote as the labeled set grows; and (iv) ranks candidates with an XGBoost ensemble trained jointly on proxy ranks and one-hot topological encodings.
  arXiv  Detail & Related papers  (2026-09-12T23:34:41Z)
• Similarity-Guided Curriculum Fine-Tuning of LLMs for Neural Architecture Synthesis [48.83701310501069]
  We introduce a MinHash-based similarity scheduling framework that constructs a progressive curriculum over neural architecture code for neural architecture search (NAS)<n>We evaluate the framework on OlympicCoder-7B within the LEMUR benchmark on CIFAR-10 image classification.<n>We report a cross-dataset transfer observation on SVHN, where direct base-model generation without curriculum yields 27% peak SR at substantially lower accuracy (60.5%) than the CIFAR-10 equivalent.
  arXiv  Detail & Related papers  (2026-07-13T14:14:30Z)
• Amplifying, Not Learning: Fine-Tuned AI Text Detectors Amplify a Pretrained Direction [51.56484100374058]
  Text detectors amplify a pretrained typicality axis; they do not construct an AI-vs-human boundary.<n>On raw encoders before any task supervision, projecting onto centroid(AI)-centroid(HC3) achieves NYT-vs-HC3 AUROC 0.806/0.944/0.834 across three architectures.<n>On RoBERTa-base, raw projection exceeds fine-tuning; on RoBERTa-base, full fine-tuning reduces discrimination below raw on both fluent-formal populations tested.
  arXiv  Detail & Related papers  (2026-05-20T19:08:38Z)
• Zero-Shot Neural Architecture Search with Weighted Response Correlation [5.953576590699768]
  We present a novel training-free estimation proxy called weighted response correlation (WRCor)<n>WRCor utilizes coefficient correlation matrices of responses across different input samples to calculate the proxy scores of estimated architectures.<n>Our NAS algorithm can discover an architecture with a 22.1% test error on the ImageNet-1k dataset within 4 GPU hours.
  arXiv  Detail & Related papers  (2025-07-08T02:19:29Z)
• Generalizing Few-Shot NAS with Gradient Matching [165.5690495295074]
  One-Shot methods train one supernet to approximate the performance of every architecture in the search space via weight-sharing. Few-Shot NAS reduces the level of weight-sharing by splitting the One-Shot supernet into multiple separated sub-supernets. It significantly outperforms its Few-Shot counterparts while surpassing previous comparable methods in terms of the accuracy of derived architectures.
  arXiv  Detail & Related papers  (2022-03-29T03:06:16Z)
• Neural Architecture Ranker [19.21631623578852]
  Architecture ranking has recently been advocated to design an efficient and effective performance predictor for Neural Architecture Search (NAS) Inspired by the stratification stratification, we propose a predictor, namely Neural Ranker (NAR)
  arXiv  Detail & Related papers  (2022-01-30T04:54:59Z)
• BaLeNAS: Differentiable Architecture Search via the Bayesian Learning Rule [95.56873042777316]
  Differentiable Architecture Search (DARTS) has received massive attention in recent years, mainly because it significantly reduces the computational cost. This paper formulates the neural architecture search as a distribution learning problem through relaxing the architecture weights into Gaussian distributions. We demonstrate how the differentiable NAS benefits from Bayesian principles, enhancing exploration and improving stability.
  arXiv  Detail & Related papers  (2021-11-25T18:13:42Z)
• ZARTS: On Zero-order Optimization for Neural Architecture Search [94.41017048659664]
  Differentiable architecture search (DARTS) has been a popular one-shot paradigm for NAS due to its high efficiency. This work turns to zero-order optimization and proposes a novel NAS scheme, called ZARTS, to search without enforcing the above approximation. In particular, results on 12 benchmarks verify the outstanding robustness of ZARTS, where the performance of DARTS collapses due to its known instability issue.
  arXiv  Detail & Related papers  (2021-10-10T09:35:15Z)
• AceNAS: Learning to Rank Ace Neural Architectures with Weak Supervision of Weight Sharing [6.171090327531059]
  We introduce Learning to Rank methods to select the best (ace) architectures from a space. We also propose to leverage weak supervision from weight sharing by pretraining architecture representation on weak labels obtained from the super-net. Experiments on NAS benchmarks and large-scale search spaces demonstrate that our approach outperforms SOTA with a significantly reduced search cost.
  arXiv  Detail & Related papers  (2021-08-06T08:31:42Z)
• Semi-Supervised Neural Architecture Search [185.0651567642238]
  SemiNAS is a semi-supervised Neural architecture search (NAS) approach that leverages numerous unlabeled architectures (without evaluation and thus nearly no cost) It achieves 94.02% test accuracy on NASBench-101, outperforming all the baselines when using the same number of architectures. It achieves 97% intelligibility rate in the low-resource setting and 15% test error rate in the robustness setting, with 9%, 7% improvements over the baseline respectively.
  arXiv  Detail & Related papers  (2020-02-24T17:23:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.