OptiProxy-NAS: Optimization Proxy based End-to-End Neural Architecture Search

• URL: http://arxiv.org/abs/2509.05656v1
• Date: Sat, 06 Sep 2025 09:26:02 GMT
• Title: OptiProxy-NAS: Optimization Proxy based End-to-End Neural Architecture Search
• Abstract summary: We propose an optimization proxy to streamline the Neural Architecture Search (NAS) problem.<n>Using a proxy representation, the NAS space is reformulated to be continuous, differentiable, and smooth.<n>Our comprehensive experiments on $12$ NAS tasks of $4$ search spaces across three different domains fully demonstrate the superior search results and efficiency.
• Score: 9.616990304130203
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Neural architecture search (NAS) is a hard computationally expensive optimization problem with a discrete, vast, and spiky search space. One of the key research efforts dedicated to this space focuses on accelerating NAS via certain proxy evaluations of neural architectures. Different from the prevalent predictor-based methods using surrogate models and differentiable architecture search via supernetworks, we propose an optimization proxy to streamline the NAS as an end-to-end optimization framework, named OptiProxy-NAS. In particular, using a proxy representation, the NAS space is reformulated to be continuous, differentiable, and smooth. Thereby, any differentiable optimization method can be applied to the gradient-based search of the relaxed architecture parameters. Our comprehensive experiments on $12$ NAS tasks of $4$ search spaces across three different domains including computer vision, natural language processing, and resource-constrained NAS fully demonstrate the superior search results and efficiency. Further experiments on low-fidelity scenarios verify the flexibility.

Related papers

• Designing Compact Neural Architectures via Neuron Gating and Mixed Activation [0.18425326702824343]
  We develop a general bilevel optimization framework for Neural Architecture Search (NAS)<n>We propose three scalable formulations that replace discrete neuron- and activation-level decisions with continuous relaxations.<n>Experiments show that the proposed methods consistently identify compact architectures with competitive or improved predictive performance.
  arXiv  Detail & Related papers  (2026-08-14T16:28:32Z)
• LM-Searcher: Cross-domain Neural Architecture Search with LLMs via Unified Numerical Encoding [55.5535016040221]
  LM-Searcher is a novel framework for cross-domain neural architecture optimization.<n>Central to our approach is NCode, a universal numerical string representation for neural architectures.<n>Our dataset, encompassing a wide range of architecture-performance pairs, encourages robust and transferable learning.
  arXiv  Detail & Related papers  (2025-09-06T09:26:39Z)
• Generalizable Lightweight Proxy for Robust NAS against Diverse Perturbations [59.683234126055694]
  Recent neural architecture search (NAS) frameworks have been successful in finding optimal architectures for given conditions. We propose a novel lightweight robust zero-cost proxy that considers the consistency across features, parameters, and gradients of both clean and perturbed images. Our approach facilitates an efficient and rapid search for neural architectures capable of learning generalizable features that exhibit robustness across diverse perturbations.
  arXiv  Detail & Related papers  (2023-06-08T08:34:26Z)
• $\eta$-DARTS++: Bi-level Regularization for Proxy-robust Differentiable Architecture Search [96.99525100285084]
  Regularization method, Beta-Decay, is proposed to regularize the DARTS-based NAS searching process (i.e., $beta$-DARTS) In-depth theoretical analyses on how it works and why it works are provided.
  arXiv  Detail & Related papers  (2023-01-16T12:30:32Z)
• BaLeNAS: Differentiable Architecture Search via the Bayesian Learning Rule [95.56873042777316]
  Differentiable Architecture Search (DARTS) has received massive attention in recent years, mainly because it significantly reduces the computational cost. This paper formulates the neural architecture search as a distribution learning problem through relaxing the architecture weights into Gaussian distributions. We demonstrate how the differentiable NAS benefits from Bayesian principles, enhancing exploration and improving stability.
  arXiv  Detail & Related papers  (2021-11-25T18:13:42Z)
• Approximate Neural Architecture Search via Operation Distribution Learning [4.358626952482686]
  We show that given an architectural cell, its performance largely depends on the ratio of used operations. This intuition is to any specific search strategy and can be applied to a diverse set of NAS algorithms.
  arXiv  Detail & Related papers  (2021-11-08T17:38:29Z)
• TND-NAS: Towards Non-differentiable Objectives in Progressive Differentiable NAS Framework [6.895590095853327]
  Differentiable architecture search has gradually become the mainstream research topic in the field of Neural Architecture Search (NAS) Recent differentiable NAS also aims at further improving the search performance and reducing the GPU-memory consumption. We propose the TND-NAS, which is with the merits of the high efficiency in differentiable NAS framework and the compatibility among non-differentiable metrics in Multi-objective NAS.
  arXiv  Detail & Related papers  (2021-11-06T14:19:36Z)
• L$^{2}$NAS: Learning to Optimize Neural Architectures via Continuous-Action Reinforcement Learning [23.25155249879658]
  Differentiable architecture search (NAS) achieved remarkable results in deep neural network design. We show that L$2$ achieves state-of-theart results on DART201 benchmark as well as NASS and Once-for-All search policies.
  arXiv  Detail & Related papers  (2021-09-25T19:26:30Z)
• iDARTS: Differentiable Architecture Search with Stochastic Implicit Gradients [75.41173109807735]
  Differentiable ARchiTecture Search (DARTS) has recently become the mainstream of neural architecture search (NAS) We tackle the hypergradient computation in DARTS based on the implicit function theorem. We show that the architecture optimisation with the proposed method, named iDARTS, is expected to converge to a stationary point.
  arXiv  Detail & Related papers  (2021-06-21T00:44:11Z)
• Trilevel Neural Architecture Search for Efficient Single Image Super-Resolution [127.92235484598811]
  This paper proposes a trilevel neural architecture search (NAS) method for efficient single image super-resolution (SR) For modeling the discrete search space, we apply a new continuous relaxation on the discrete search spaces to build a hierarchical mixture of network-path, cell-operations, and kernel-width. An efficient search algorithm is proposed to perform optimization in a hierarchical supernet manner.
  arXiv  Detail & Related papers  (2021-01-17T12:19:49Z)
• Smooth Variational Graph Embeddings for Efficient Neural Architecture Search [41.62970837629573]
  We propose a two-sided variational graph autoencoder, which allows to smoothly encode and accurately reconstruct neural architectures from various search spaces. We evaluate the proposed approach on neural architectures defined by the ENAS approach, the NAS-Bench-101 and the NAS-Bench-201 search spaces.
  arXiv  Detail & Related papers  (2020-10-09T17:05:41Z)
• Geometry-Aware Gradient Algorithms for Neural Architecture Search [41.943045315986744]
  We argue for the study of single-level empirical risk minimization to understand NAS with weight-sharing. We present a geometry-aware framework that exploits the underlying structure of this optimization to return sparse architectural parameters. We achieve state-of-the-art accuracy on the latest NAS benchmarks in computer vision.
  arXiv  Detail & Related papers  (2020-04-16T17:46:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.