Demystifying Map Space Exploration for NPUs

• URL: http://arxiv.org/abs/2210.03731v1
• Date: Fri, 7 Oct 2022 17:58:45 GMT
• Title: Demystifying Map Space Exploration for NPUs
• Authors: Sheng-Chun Kao, Angshuman Parashar, Po-An Tsai, Tushar Krishna
• Abstract summary: Map Space Exploration is the problem of finding optimized mappings of a Deep Neural Network (DNN) model. We do a first-of-its-kind apples-to-apples comparison of search techniques leveraged by different mappers. Next, we propose two new techniques that can augment existing mappers.
• Score: 4.817475305740601
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Map Space Exploration is the problem of finding optimized mappings of a Deep Neural Network (DNN) model on an accelerator. It is known to be extremely computationally expensive, and there has been active research looking at both heuristics and learning-based methods to make the problem computationally tractable. However, while there are dozens of mappers out there (all empirically claiming to find better mappings than others), the research community lacks systematic insights on how different search techniques navigate the map-space and how different mapping axes contribute to the accelerator's performance and efficiency. Such insights are crucial to developing mapping frameworks for emerging DNNs that are increasingly irregular (due to neural architecture search) and sparse, making the corresponding map spaces much more complex. In this work, rather than proposing yet another mapper, we do a first-of-its-kind apples-to-apples comparison of search techniques leveraged by different mappers. Next, we extract the learnings from our study and propose two new techniques that can augment existing mappers -- warm-start and sparsity-aware -- that demonstrate speedups, scalability, and robustness across diverse DNN models.

Related papers

• FRAME: A Modular Framework for Autonomous Map Merging: Advancements in the Field [12.247977717070773]
  This article presents a novel approach for merging 3D point cloud maps in the context of egocentric multi-robot exploration. The proposed approach leverages state-of-the-art place recognition and learned descriptors to efficiently detect overlap between maps. The effectiveness of the proposed framework is successfully demonstrated through multiple field missions of robot exploration.
  arXiv  Detail & Related papers  (2024-04-27T20:54:15Z)
• Active Neural Topological Mapping for Multi-Agent Exploration [24.91397816926568]
  Multi-agent cooperative exploration problem requires multiple agents to explore an unseen environment via sensory signals in a limited time. Topological maps are a promising alternative as they consist only of nodes and edges with abstract but essential information. Deep reinforcement learning has shown great potential for learning (near) optimal policies through fast end-to-end inference. We propose Multi-Agent Neural Topological Mapping (MANTM) to improve exploration efficiency and generalization for multi-agent exploration tasks.
  arXiv  Detail & Related papers  (2023-11-01T03:06:14Z)
• MultiScale Probability Map guided Index Pooling with Attention-based learning for Road and Building Segmentation [18.838213902873616]
  We propose a novel attention-aware segmentation framework, Multi-Scale Supervised Dilated Multiple-Path Attention Network (MSSDMPA-Net) MSSDMPA-Net is equipped with two new modules Dynamic Attention Map Guided Index Pooling (DAMIP) and Dynamic Attention Map Guided Spatial and Channel Attention (DAMSCA) to precisely extract the building footprints and road maps from remotely sensed images.
  arXiv  Detail & Related papers  (2023-02-18T19:57:25Z)
• Searching a High-Performance Feature Extractor for Text Recognition Network [92.12492627169108]
  We design a domain-specific search space by exploring principles for having good feature extractors. As the space is huge and complexly structured, no existing NAS algorithms can be applied. We propose a two-stage algorithm to effectively search in the space.
  arXiv  Detail & Related papers  (2022-09-27T03:49:04Z)
• SESS: Saliency Enhancing with Scaling and Sliding [42.188013259368766]
  High-quality saliency maps are essential in several machine learning application areas including explainable AI and weakly supervised object detection and segmentation. We propose a novel saliency enhancing approach called SESS (Saliency Enhancing with Scaling and Sliding) It is a method and model extension to existing saliency map generation methods.
  arXiv  Detail & Related papers  (2022-07-05T02:16:23Z)
• Edge-featured Graph Neural Architecture Search [131.4361207769865]
  We propose Edge-featured Graph Neural Architecture Search to find the optimal GNN architecture. Specifically, we design rich entity and edge updating operations to learn high-order representations. We show EGNAS can search better GNNs with higher performance than current state-of-the-art human-designed and searched-based GNNs.
  arXiv  Detail & Related papers  (2021-09-03T07:53:18Z)
• Fast discovery of multidimensional subsequences for robust trajectory classification [0.2578242050187029]
  Trajectory classification tasks became more complex as large volumes of mobility data are being generated every day. Fast classification algorithms are essential for discovering knowledge in trajectory data for real applications. We propose a method for fast discovery of subtrajectories with the reduction of the search space and the optimization of the MASTERMovelets method.
  arXiv  Detail & Related papers  (2021-02-09T11:54:33Z)
• CATCH: Context-based Meta Reinforcement Learning for Transferrable Architecture Search [102.67142711824748]
  CATCH is a novel Context-bAsed meTa reinforcement learning algorithm for transferrable arChitecture searcH. The combination of meta-learning and RL allows CATCH to efficiently adapt to new tasks while being agnostic to search spaces. It is also capable of handling cross-domain architecture search as competitive networks on ImageNet, COCO, and Cityscapes are identified.
  arXiv  Detail & Related papers  (2020-07-18T09:35:53Z)
• A Survey on Deep Learning for Localization and Mapping: Towards the Age of Spatial Machine Intelligence [48.67755344239951]
  We provide a comprehensive survey, and propose a new taxonomy for localization and mapping using deep learning. A wide range of topics are covered, from learning odometry estimation, mapping, to global localization and simultaneous localization and mapping. It is our hope that this work can connect emerging works from robotics, computer vision and machine learning communities.
  arXiv  Detail & Related papers  (2020-06-22T19:01:21Z)
• DrNAS: Dirichlet Neural Architecture Search [88.56953713817545]
  We treat the continuously relaxed architecture mixing weight as random variables, modeled by Dirichlet distribution. With recently developed pathwise derivatives, the Dirichlet parameters can be easily optimized with gradient-based generalization. To alleviate the large memory consumption of differentiable NAS, we propose a simple yet effective progressive learning scheme.
  arXiv  Detail & Related papers  (2020-06-18T08:23:02Z)
• AutoML-Zero: Evolving Machine Learning Algorithms From Scratch [76.83052807776276]
  We show that it is possible to automatically discover complete machine learning algorithms just using basic mathematical operations as building blocks. We demonstrate this by introducing a novel framework that significantly reduces human bias through a generic search space. We believe these preliminary successes in discovering machine learning algorithms from scratch indicate a promising new direction in the field.
  arXiv  Detail & Related papers  (2020-03-06T19:00:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.