AttendSeg: A Tiny Attention Condenser Neural Network for Semantic Segmentation on the Edge

• URL: http://arxiv.org/abs/2104.14623v1
• Date: Thu, 29 Apr 2021 19:19:04 GMT
• Title: AttendSeg: A Tiny Attention Condenser Neural Network for Semantic Segmentation on the Edge
• Authors: Xiaoyu Wen, Mahmoud Famouri, Andrew Hryniowski, and Alexander Wong
• Abstract summary: We introduce textbfAttendSeg, a low-precision, highly compact deep neural network tailored for on-device semantic segmentation. AttendSeg possesses a self-attention network architecture comprising of light-weight attention condensers for improved spatial-channel selective attention.
• Score: 71.80459780697956
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this study, we introduce \textbf{AttendSeg}, a low-precision, highly compact deep neural network tailored for on-device semantic segmentation. AttendSeg possesses a self-attention network architecture comprising of light-weight attention condensers for improved spatial-channel selective attention at a very low complexity. The unique macro-architecture and micro-architecture design properties of AttendSeg strike a strong balance between representational power and efficiency, achieved via a machine-driven design exploration strategy tailored specifically for the task at hand. Experimental results demonstrated that the proposed AttendSeg can achieve segmentation accuracy comparable to much larger deep neural networks with greater complexity while possessing a significantly lower architecture and computational complexity (requiring as much as >27x fewer MACs, >72x fewer parameters, and >288x lower weight memory requirements), making it well-suited for TinyML applications on the edge.

Related papers

• MFPNet: Multi-scale Feature Propagation Network For Lightweight Semantic Segmentation [5.58363644107113]
  We propose a novel lightweight segmentation architecture, called Multi-scale Feature Propagation Network (Net) We design a robust-Decoder structure featuring symmetrical residual blocks that consist of flexible bottleneck residual modules (BRMs) Taking benefit of their capacity to model latent long-range contextual relationships, we leverage Graph Convolutional Networks (GCNs) to facilitate multiscale feature propagation between the BRM blocks.
  arXiv  Detail & Related papers  (2023-09-10T02:02:29Z)
• Faster Attention Is What You Need: A Fast Self-Attention Neural Network Backbone Architecture for the Edge via Double-Condensing Attention Condensers [71.40595908386477]
  We introduce a new faster attention condenser design called double-condensing attention condensers. The resulting backbone (which we name AttendNeXt) achieves significantly higher inference throughput on an embedded ARM processor. These promising results demonstrate that exploring different efficient architecture designs and self-attention mechanisms can lead to interesting new building blocks for TinyML applications.
  arXiv  Detail & Related papers  (2022-08-15T02:47:33Z)
• ME-CapsNet: A Multi-Enhanced Capsule Networks with Routing Mechanism [0.0]
  This research focuses on bringing in a novel solution that uses sophisticated optimization for enhancing both the spatial and channel components inside each layer's receptive field. We have proposed ME-CapsNet by introducing deeper convolutional layers to extract important features before passing through modules of capsule layers strategically. The deeper convolutional layer includes blocks of Squeeze-Excitation networks which use a sampling approach for reconstructing their interdependencies without much loss of important feature information.
  arXiv  Detail & Related papers  (2022-03-29T13:29:38Z)
• AttendNets: Tiny Deep Image Recognition Neural Networks for the Edge via Visual Attention Condensers [81.17461895644003]
  We introduce AttendNets, low-precision, highly compact deep neural networks tailored for on-device image recognition. AttendNets possess deep self-attention architectures based on visual attention condensers. Results show AttendNets have significantly lower architectural and computational complexity when compared to several deep neural networks.
  arXiv  Detail & Related papers  (2020-09-30T01:53:17Z)
• TinySpeech: Attention Condensers for Deep Speech Recognition Neural Networks on Edge Devices [71.68436132514542]
  We introduce the concept of attention condensers for building low-footprint, highly-efficient deep neural networks for on-device speech recognition on the edge. To illustrate its efficacy, we introduce TinySpeech, low-precision deep neural networks tailored for on-device speech recognition.
  arXiv  Detail & Related papers  (2020-08-10T16:34:52Z)
• Real-time Semantic Segmentation with Fast Attention [94.88466483540692]
  We propose a novel architecture for semantic segmentation of high-resolution images and videos in real-time. The proposed architecture relies on our fast spatial attention, which is a simple yet efficient modification of the popular self-attention mechanism. We show that results on multiple datasets demonstrate superior performance with better accuracy and speed compared to existing approaches.
  arXiv  Detail & Related papers  (2020-07-07T22:37:16Z)
• When Residual Learning Meets Dense Aggregation: Rethinking the Aggregation of Deep Neural Networks [57.0502745301132]
  We propose Micro-Dense Nets, a novel architecture with global residual learning and local micro-dense aggregations. Our micro-dense block can be integrated with neural architecture search based models to boost their performance.
  arXiv  Detail & Related papers  (2020-04-19T08:34:52Z)
• DVNet: A Memory-Efficient Three-Dimensional CNN for Large-Scale Neurovascular Reconstruction [1.9199289015460215]
  We present a fully-convolutional, deep, and densely-connected encoder-decoder for pixel-wise semantic segmentation. The proposed network provides superior performance for semantic segmentation problems applied to open-source benchmarks.
  arXiv  Detail & Related papers  (2020-02-04T22:39:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.