AutoML for neuromorphic computing and application-driven co-design: asynchronous, massively parallel optimization of spiking architectures

• URL: http://arxiv.org/abs/2302.13210v1
• Date: Sun, 26 Feb 2023 02:26:45 GMT
• Title: AutoML for neuromorphic computing and application-driven co-design: asynchronous, massively parallel optimization of spiking architectures
• Authors: Angel Yanguas-Gil and Sandeep Madireddy
• Abstract summary: We have extended AutoML inspired approaches to the exploration and optimization of neuromorphic architectures. We are able to efficiently explore the configuration space of neuromorphic architectures and identify the subset of conditions leading to the highest performance.
• Score: 3.8937756915387505
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this work we have extended AutoML inspired approaches to the exploration and optimization of neuromorphic architectures. Through the integration of a parallel asynchronous model-based search approach with a simulation framework to simulate spiking architectures, we are able to efficiently explore the configuration space of neuromorphic architectures and identify the subset of conditions leading to the highest performance in a targeted application. We have demonstrated this approach on an exemplar case of real time, on-chip learning application. Our results indicate that we can effectively use optimization approaches to optimize complex architectures, therefore providing a viable pathway towards application-driven codesign.

Related papers

• EPS-MoE: Expert Pipeline Scheduler for Cost-Efficient MoE Inference [49.94169109038806]
  This paper introduces EPS-MoE, a novel expert pipeline scheduler for MoE. Our results demonstrate an average 21% improvement in prefill throughput over existing parallel inference methods.
  arXiv  Detail & Related papers  (2024-10-16T05:17:49Z)
• Mechanistic Design and Scaling of Hybrid Architectures [114.3129802943915]
  We identify and test new hybrid architectures constructed from a variety of computational primitives. We experimentally validate the resulting architectures via an extensive compute-optimal and a new state-optimal scaling law analysis. We find MAD synthetics to correlate with compute-optimal perplexity, enabling accurate evaluation of new architectures.
  arXiv  Detail & Related papers  (2024-03-26T16:33:12Z)
• Introducing Interactions in Multi-Objective Optimization of Software Architectures [2.920908475492581]
  This study investigates the impact of designer interactions on software architecture optimization. By directing the search towards regions of interest, the interaction uncovers architectures that remain unexplored in the fully automated process.
  arXiv  Detail & Related papers  (2023-08-29T07:49:46Z)
• CR-LSO: Convex Neural Architecture Optimization in the Latent Space of Graph Variational Autoencoder with Input Convex Neural Networks [7.910915721525413]
  In neural architecture search (NAS) methods based on latent space optimization (LSO), a deep generative model is trained to embed discrete neural architectures into a continuous latent space. This paper develops a convexity architecture regularized space (CRLSO) method, which aims to regularize the learning process of space in order to obtain a convex performance mapping. Experimental results on three popular NAS benchmarks show that CR-LSO achieves competitive evaluation results in terms of both computational complexity and performance.
  arXiv  Detail & Related papers  (2022-11-11T01:55:11Z)
• Slimmable Domain Adaptation [112.19652651687402]
  We introduce a simple framework, Slimmable Domain Adaptation, to improve cross-domain generalization with a weight-sharing model bank. Our framework surpasses other competing approaches by a very large margin on multiple benchmarks.
  arXiv  Detail & Related papers  (2022-06-14T06:28:04Z)
• Learning Where To Look -- Generative NAS is Surprisingly Efficient [11.83842808044211]
  We propose a generative model, paired with a surrogate predictor, that iteratively learns to generate samples from increasingly promising latent subspaces. This approach leads to very effective and efficient architecture search, while keeping the query amount low.
  arXiv  Detail & Related papers  (2022-03-16T16:27:11Z)
• Dynamically Grown Generative Adversarial Networks [111.43128389995341]
  We propose a method to dynamically grow a GAN during training, optimizing the network architecture and its parameters together with automation. The method embeds architecture search techniques as an interleaving step with gradient-based training to periodically seek the optimal architecture-growing strategy for the generator and discriminator.
  arXiv  Detail & Related papers  (2021-06-16T01:25:51Z)
• Operation Embeddings for Neural Architecture Search [15.033712726016255]
  We propose the replacement of fixed operator encoding with learnable representations in the optimization process. Our method produces top-performing architectures that share similar operation and graph patterns.
  arXiv  Detail & Related papers  (2021-05-11T09:17:10Z)
• Towards Accurate and Compact Architectures via Neural Architecture Transformer [95.4514639013144]
  It is necessary to optimize the operations inside an architecture to improve the performance without introducing extra computational cost. We have proposed a Neural Architecture Transformer (NAT) method which casts the optimization problem into a Markov Decision Process (MDP) We propose a Neural Architecture Transformer++ (NAT++) method which further enlarges the set of candidate transitions to improve the performance of architecture optimization.
  arXiv  Detail & Related papers  (2021-02-20T09:38:10Z)
• Apollo: Transferable Architecture Exploration [26.489275442359464]
  We propose a transferable architecture exploration framework, dubbed Apollo. We show that our framework finds high reward design configurations more sample-efficiently than a baseline black-box optimization approach.
  arXiv  Detail & Related papers  (2021-02-02T19:36:02Z)
• Off-Policy Reinforcement Learning for Efficient and Effective GAN Architecture Search [50.40004966087121]
  We introduce a new reinforcement learning based neural architecture search (NAS) methodology for generative adversarial network (GAN) architecture search. The key idea is to formulate the GAN architecture search problem as a Markov decision process (MDP) for smoother architecture sampling. We exploit an off-policy GAN architecture search algorithm that makes efficient use of the samples generated by previous policies.
  arXiv  Detail & Related papers  (2020-07-17T18:29:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.