AIRCHITECT: Learning Custom Architecture Design and Mapping Space
- URL: http://arxiv.org/abs/2108.08295v1
- Date: Mon, 16 Aug 2021 05:05:52 GMT
- Title: AIRCHITECT: Learning Custom Architecture Design and Mapping Space
- Authors: Ananda Samajdar, Jan Moritz Joseph, Matthew Denton, Tushar Krishna
- Abstract summary: We train a machine learning model to predict optimal parameters for the design and mapping space of custom architectures.
We show that it is possible to capture the design space and train a model to "generalize" prediction the optimal design and mapping parameters.
We train a custom network architecture called AIRCHITECT, which is capable of learning the architecture design space with as high as 94.3% test accuracy.
- Score: 2.498907460918493
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Design space exploration is an important but costly step involved in the
design/deployment of custom architectures to squeeze out maximum possible
performance and energy efficiency. Conventionally, optimizations require
iterative sampling of the design space using simulation or heuristic tools. In
this paper we investigate the possibility of learning the optimization task
using machine learning and hence using the learnt model to predict optimal
parameters for the design and mapping space of custom architectures, bypassing
any exploration step. We use three case studies involving the optimal array
design, SRAM buffer sizing, mapping, and schedule determination for
systolic-array-based custom architecture design and mapping space. Within the
purview of these case studies, we show that it is possible to capture the
design space and train a model to "generalize" prediction the optimal design
and mapping parameters when queried with workload and design constraints. We
perform systematic design-aware and statistical analysis of the optimization
space for our case studies and highlight the patterns in the design space. We
formulate the architecture design and mapping as a machine learning problem
that allows us to leverage existing ML models for training and inference. We
design and train a custom network architecture called AIRCHITECT, which is
capable of learning the architecture design space with as high as 94.3% test
accuracy and predicting optimal configurations which achieve on average
(GeoMean) of 99.9% the best possible performance on a test dataset with $10^5$
GEMM workloads.
Related papers
- Automated architectural space layout planning using a physics-inspired generative design framework [4.202451453254076]
The determination of space layout is one of the primary activities in the schematic design stage of an architectural project.
The proposed approach integrates a novel physics-inspired parametric model for space layout planning and an evolutionary optimisation metaheuristic.
Results revealed that such a generative design framework can generate a wide variety of design suggestions at the schematic design stage.
arXiv Detail & Related papers (2024-06-21T02:50:52Z) - Implicitly Guided Design with PropEn: Match your Data to Follow the Gradient [52.2669490431145]
PropEn is inspired by'matching', which enables implicit guidance without training a discriminator.
We show that training with a matched dataset approximates the gradient of the property of interest while remaining within the data distribution.
arXiv Detail & Related papers (2024-05-28T11:30:19Z) - Mechanistic Design and Scaling of Hybrid Architectures [114.3129802943915]
We identify and test new hybrid architectures constructed from a variety of computational primitives.
We experimentally validate the resulting architectures via an extensive compute-optimal and a new state-optimal scaling law analysis.
We find MAD synthetics to correlate with compute-optimal perplexity, enabling accurate evaluation of new architectures.
arXiv Detail & Related papers (2024-03-26T16:33:12Z) - Compositional Generative Inverse Design [69.22782875567547]
Inverse design, where we seek to design input variables in order to optimize an underlying objective function, is an important problem.
We show that by instead optimizing over the learned energy function captured by the diffusion model, we can avoid such adversarial examples.
In an N-body interaction task and a challenging 2D multi-airfoil design task, we demonstrate that by composing the learned diffusion model at test time, our method allows us to design initial states and boundary shapes.
arXiv Detail & Related papers (2024-01-24T01:33:39Z) - Optimization for truss design using Bayesian optimization [1.5070398746522742]
The shape of the truss is a dominant factor in determining the capacity of load it can bear.
At a given parameter space, our goal is to find the parameters of a hull that maximize the load-bearing capacity and also don't yield to the induced stress.
We rely on finite element analysis, which is a computationally costly design analysis tool for design evaluation.
arXiv Detail & Related papers (2023-05-27T10:28:27Z) - Design Space Exploration and Explanation via Conditional Variational
Autoencoders in Meta-model-based Conceptual Design of Pedestrian Bridges [52.77024349608834]
This paper provides a performance-driven design exploration framework to augment the human designer through a Conditional Variational Autoencoder (CVAE)
The CVAE is trained on 18'000 synthetically generated instances of a pedestrian bridge in Switzerland.
arXiv Detail & Related papers (2022-11-29T17:28:31Z) - Efficient Automatic Machine Learning via Design Graphs [72.85976749396745]
We propose FALCON, an efficient sample-based method to search for the optimal model design.
FALCON features 1) a task-agnostic module, which performs message passing on the design graph via a Graph Neural Network (GNN), and 2) a task-specific module, which conducts label propagation of the known model performance information.
We empirically show that FALCON can efficiently obtain the well-performing designs for each task using only 30 explored nodes.
arXiv Detail & Related papers (2022-10-21T21:25:59Z) - A Semi-Decoupled Approach to Fast and Optimal Hardware-Software
Co-Design of Neural Accelerators [22.69558355718029]
Hardware-software co-design has been emerging to fully reap the benefits of flexible design spaces and optimize neural network performance.
Such co-design enlarges the total search space to practically infinity and presents substantial challenges.
We propose a emphsemi-decoupled approach to reduce the size of the total design space by orders of magnitude, yet without losing optimality.
arXiv Detail & Related papers (2022-03-25T21:49:42Z) - Apollo: Transferable Architecture Exploration [26.489275442359464]
We propose a transferable architecture exploration framework, dubbed Apollo.
We show that our framework finds high reward design configurations more sample-efficiently than a baseline black-box optimization approach.
arXiv Detail & Related papers (2021-02-02T19:36:02Z) - Evolving Search Space for Neural Architecture Search [70.71153433676024]
We present a Neural Search-space Evolution (NSE) scheme that amplifies the results from the previous effort by maintaining an optimized search space subset.
We achieve 77.3% top-1 retrain accuracy on ImageNet with 333M FLOPs, which yielded a state-of-the-art performance.
When the latency constraint is adopted, our result also performs better than the previous best-performing mobile models with a 77.9% Top-1 retrain accuracy.
arXiv Detail & Related papers (2020-11-22T01:11:19Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.