Predictor-Free and Hardware-Aware Federated Neural Architecture Search via Pareto-Guided Supernet Training

• URL: http://arxiv.org/abs/2601.15127v2
• Date: Wed, 28 Jan 2026 13:58:23 GMT
• Title: Predictor-Free and Hardware-Aware Federated Neural Architecture Search via Pareto-Guided Supernet Training
• Authors: Bostan Khan, Masoud Daneshtalab,
• Abstract summary: Neural Architecture Search (FedNAS) aims to automate model design for privacy-preserving Learning (FL)<n>FedNAS currently faces two critical bottlenecks: unguided supernet training that yields suboptimal models, and costly multi-hour pipelines for post-training discovery.<n>We introduce DeepFedNAS, a novel, two-phase framework underpinned by a multi-objective fitness function that synthesizes architectural Federateds.<n>DeepFedNAS makes hardware-aware FL deployments instantaneous and practical.
• Score: 2.1485350418225244
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Federated Neural Architecture Search (FedNAS) aims to automate model design for privacy-preserving Federated Learning (FL) but currently faces two critical bottlenecks: unguided supernet training that yields suboptimal models, and costly multi-hour pipelines for post-training subnet discovery. We introduce DeepFedNAS, a novel, two-phase framework underpinned by a multi-objective fitness function that synthesizes mathematical network design with architectural heuristics. Enabled by a re-engineered supernet, DeepFedNAS introduces Federated Pareto Optimal Supernet Training, which leverages a pre-computed Pareto-optimal cache of high-fitness architectures as an intelligent curriculum to optimize shared supernet weights. Subsequently, its Predictor-Free Search Method eliminates the need for costly accuracy surrogates by utilizing this fitness function as a direct, zero-cost proxy for accuracy, enabling on-demand subnet discovery in mere seconds. DeepFedNAS achieves state-of-the-art accuracy (e.g., up to 1.21% absolute improvement on CIFAR-100), superior parameter and communication efficiency, and a substantial ~61x speedup in total post-training search pipeline time. By reducing the pipeline from over 20 hours to approximately 20 minutes (including initial cache generation) and enabling 20-second individual subnet searches, DeepFedNAS makes hardware-aware FL deployments instantaneous and practical. The complete source code and experimental scripts are available at: https://github.com/bostankhan6/DeepFedNAS

Related papers

• HyperNAS: Enhancing Architecture Representation for NAS Predictor via Hypernetwork [37.904207059004385]
  HyperNAS is a novel neural predictor paradigm for enhancing architecture representation learning.<n>We show that HyperNAS strikes new state-of-the-art results, with 97.60% top-1 accuracy on CIFAR-10 and 82.4% top-1 accuracy on ImageNet, using at least 5.0$times$ fewer samples.
  arXiv  Detail & Related papers  (2025-09-16T11:49:12Z)
• SalNAS: Efficient Saliency-prediction Neural Architecture Search with self-knowledge distillation [7.625269122161064]
  Recent advancements in deep convolutional neural networks have significantly improved the performance of saliency prediction. We propose a new Neural Architecture Search framework for saliency prediction with two contributions. By utilizing Self-KD, SalNAS outperforms other state-of-the-art saliency prediction models in most evaluation rubrics.
  arXiv  Detail & Related papers  (2024-07-29T14:48:34Z)
• Mixture-of-Supernets: Improving Weight-Sharing Supernet Training with Architecture-Routed Mixture-of-Experts [55.470959564665705]
  Weight-sharing supernets are crucial for performance estimation in cutting-edge neural search frameworks. The proposed method attains state-of-the-art (SoTA) performance in NAS for fast machine translation models. It excels in NAS for building memory-efficient task-agnostic BERT models.
  arXiv  Detail & Related papers  (2023-06-08T00:35:36Z)
• Towards Self-supervised and Weight-preserving Neural Architecture Search [38.497608743382145]
  We propose the self-supervised and weight-preserving neural architecture search (SSWP-NAS) as an extension of the current NAS framework. Experiments show that the architectures searched by the proposed framework achieve state-of-the-art accuracy on CIFAR-10, CIFAR-100, and ImageNet datasets.
  arXiv  Detail & Related papers  (2022-06-08T18:48:05Z)
• L$^{2}$NAS: Learning to Optimize Neural Architectures via Continuous-Action Reinforcement Learning [23.25155249879658]
  Differentiable architecture search (NAS) achieved remarkable results in deep neural network design. We show that L$2$ achieves state-of-theart results on DART201 benchmark as well as NASS and Once-for-All search policies.
  arXiv  Detail & Related papers  (2021-09-25T19:26:30Z)
• BN-NAS: Neural Architecture Search with Batch Normalization [116.47802796784386]
  We present BN-NAS, neural architecture search with Batch Normalization (BN-NAS), to accelerate neural architecture search (NAS) BN-NAS can significantly reduce the time required by model training and evaluation in NAS.
  arXiv  Detail & Related papers  (2021-08-16T23:23:21Z)
• FNAS: Uncertainty-Aware Fast Neural Architecture Search [54.49650267859032]
  Reinforcement learning (RL)-based neural architecture search (NAS) generally guarantees better convergence yet suffers from the requirement of huge computational resources. We propose a general pipeline to accelerate the convergence of the rollout process as well as the RL process in NAS. Experiments on the Mobile Neural Architecture Search (MNAS) search space show the proposed Fast Neural Architecture Search (FNAS) accelerates standard RL-based NAS process by 10x.
  arXiv  Detail & Related papers  (2021-05-25T06:32:52Z)
• Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective [88.39981851247727]
  We propose a novel framework called training-free neural architecture search (TE-NAS) TE-NAS ranks architectures by analyzing the spectrum of the neural tangent kernel (NTK) and the number of linear regions in the input space. We show that: (1) these two measurements imply the trainability and expressivity of a neural network; (2) they strongly correlate with the network's test accuracy.
  arXiv  Detail & Related papers  (2021-02-23T07:50:44Z)
• FBNetV3: Joint Architecture-Recipe Search using Predictor Pretraining [65.39532971991778]
  We present an accuracy predictor that scores architecture and training recipes jointly, guiding both sample selection and ranking. We run fast evolutionary searches in just CPU minutes to generate architecture-recipe pairs for a variety of resource constraints. FBNetV3 makes up a family of state-of-the-art compact neural networks that outperform both automatically and manually-designed competitors.
  arXiv  Detail & Related papers  (2020-06-03T05:20:21Z)
• DDPNAS: Efficient Neural Architecture Search via Dynamic Distribution Pruning [135.27931587381596]
  We propose an efficient and unified NAS framework termed DDPNAS via dynamic distribution pruning. In particular, we first sample architectures from a joint categorical distribution. Then the search space is dynamically pruned and its distribution is updated every few epochs. With the proposed efficient network generation method, we directly obtain the optimal neural architectures on given constraints.
  arXiv  Detail & Related papers  (2019-05-28T06:35:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.