Neural Architecture Search From Task Similarity Measure

• URL: http://arxiv.org/abs/2103.00241v2
• Date: Wed, 3 Mar 2021 14:53:53 GMT
• Title: Neural Architecture Search From Task Similarity Measure
• Authors: Cat P. Le, Mohammadreza Soltani, Robert Ravier, Vahid Tarokh
• Abstract summary: We propose a neural architecture search framework based on a similarity measure between various tasks defined in terms of Fisher information. By utilizing the relation between a target and a set of existing tasks, the search space of architectures can be significantly reduced.
• Score: 28.5184196829547
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this paper, we propose a neural architecture search framework based on a similarity measure between various tasks defined in terms of Fisher information. By utilizing the relation between a target and a set of existing tasks, the search space of architectures can be significantly reduced, making the discovery of the best candidates in the set of possible architectures tractable. This method eliminates the requirement for training the networks from scratch for the target task. Simulation results illustrate the efficacy of our proposed approach and its competitiveness with state-of-the-art methods.

Related papers

• Surrogate-assisted Multi-objective Neural Architecture Search for Real-time Semantic Segmentation [11.866947846619064]
  neural architecture search (NAS) has emerged as a promising avenue toward automating the design of architectures. We propose a surrogate-assisted multi-objective method to address the challenges of applying NAS to semantic segmentation. Our method can identify architectures significantly outperforming existing state-of-the-art architectures designed both manually by human experts and automatically by other NAS methods.
  arXiv  Detail & Related papers  (2022-08-14T10:18:51Z)
• Search Space Adaptation for Differentiable Neural Architecture Search in Image Classification [15.641353388251465]
  Differentiable neural architecture search (NAS) has a great impact by reducing the search cost to the level of training a single network. In this paper, we propose an adaptation scheme of the search space by introducing a search scope. The effectiveness of proposed method is demonstrated with ProxylessNAS for the image classification task.
  arXiv  Detail & Related papers  (2022-06-05T05:27:12Z)
• A Framework for Verifiable and Auditable Federated Anomaly Detection [3.639790324866155]
  Federated Leaning is an emerging approach to manage cooperation between a group of agents for the solution of Machine Learning tasks. We present a novel algorithmic architecture that tackle this problem in the particular case of Anomaly Detection.
  arXiv  Detail & Related papers  (2022-03-15T11:34:02Z)
• RANK-NOSH: Efficient Predictor-Based Architecture Search via Non-Uniform Successive Halving [74.61723678821049]
  We propose NOn-uniform Successive Halving (NOSH), a hierarchical scheduling algorithm that terminates the training of underperforming architectures early to avoid wasting budget. We formulate predictor-based architecture search as learning to rank with pairwise comparisons. The resulting method - RANK-NOSH, reduces the search budget by 5x while achieving competitive or even better performance than previous state-of-the-art predictor-based methods on various spaces and datasets.
  arXiv  Detail & Related papers  (2021-08-18T07:45:21Z)
• Elastic Architecture Search for Diverse Tasks with Different Resources [87.23061200971912]
  We study a new challenging problem of efficient deployment for diverse tasks with different resources, where the resource constraint and task of interest corresponding to a group of classes are dynamically specified at testing time. Previous NAS approaches seek to design architectures for all classes simultaneously, which may not be optimal for some individual tasks. We present a novel and general framework, called Elastic Architecture Search (EAS), permitting instant specializations at runtime for diverse tasks with various resource constraints.
  arXiv  Detail & Related papers  (2021-08-03T00:54:27Z)
• Neural Architecture Search From Fr\'echet Task Distance [50.9995960884133]
  We show how the distance between a target task and each task in a given set of baseline tasks can be used to reduce the neural architecture search space for the target task. The complexity reduction in search space for task-specific architectures is achieved by building on the optimized architectures for similar tasks instead of doing a full search without using this side information.
  arXiv  Detail & Related papers  (2021-03-23T20:43:31Z)
• Task-Aware Neural Architecture Search [33.11791812491669]
  We propose a novel framework for neural architecture search, utilizing a dictionary of models of base tasks and the similarity between the target task and the atoms of the dictionary. By introducing a gradient-based search algorithm, we can evaluate and discover the best architecture in the search space without fully training the networks.
  arXiv  Detail & Related papers  (2020-10-27T00:10:40Z)
• NAS-Navigator: Visual Steering for Explainable One-Shot Deep Neural Network Synthesis [53.106414896248246]
  We present a framework that allows analysts to effectively build the solution sub-graph space and guide the network search by injecting their domain knowledge. Applying this technique in an iterative manner allows analysts to converge to the best performing neural network architecture for a given application.
  arXiv  Detail & Related papers  (2020-09-28T01:48:45Z)
• Automated Search for Resource-Efficient Branched Multi-Task Networks [81.48051635183916]
  We propose a principled approach, rooted in differentiable neural architecture search, to automatically define branching structures in a multi-task neural network. We show that our approach consistently finds high-performing branching structures within limited resource budgets.
  arXiv  Detail & Related papers  (2020-08-24T09:49:19Z)
• MTL-NAS: Task-Agnostic Neural Architecture Search towards General-Purpose Multi-Task Learning [71.90902837008278]
  We propose to incorporate neural architecture search (NAS) into general-purpose multi-task learning (GP-MTL) In order to adapt to different task combinations, we disentangle the GP-MTL networks into single-task backbones. We also propose a novel single-shot gradient-based search algorithm that closes the performance gap between the searched architectures.
  arXiv  Detail & Related papers  (2020-03-31T09:49:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.