Enabling Capsule Networks at the Edge through Approximate Softmax and Squash Operations

• URL: http://arxiv.org/abs/2206.10200v1
• Date: Tue, 21 Jun 2022 08:57:16 GMT
• Title: Enabling Capsule Networks at the Edge through Approximate Softmax and Squash Operations
• Authors: Alberto Marchisio and Beatrice Bussolino and Edoardo Salvati and Maurizio Martina and Guido Masera and Muhammad Shafique
• Abstract summary: We propose to leverage approximate computing for designing approximate variants of the complex operations like softmax and squash. In our experiments, we evaluate tradeoffs between area, power consumption, and critical path delay of the designs implemented with the ASIC design flow.
• Score: 11.82717649039321
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Complex Deep Neural Networks such as Capsule Networks (CapsNets) exhibit high learning capabilities at the cost of compute-intensive operations. To enable their deployment on edge devices, we propose to leverage approximate computing for designing approximate variants of the complex operations like softmax and squash. In our experiments, we evaluate tradeoffs between area, power consumption, and critical path delay of the designs implemented with the ASIC design flow, and the accuracy of the quantized CapsNets, compared to the exact functions.

Related papers

• QuartDepth: Post-Training Quantization for Real-Time Depth Estimation on the Edge [55.75103034526652]
  We propose QuartDepth which adopts post-training quantization to quantize MDE models with hardware accelerations for ASICs. Our approach involves quantizing both weights and activations to 4-bit precision, reducing the model size and computation cost. We design a flexible and programmable hardware accelerator by supporting kernel fusion and customized instruction programmability.
  arXiv  Detail & Related papers  (2025-03-20T21:03:10Z)
• Node Centrality Approximation For Large Networks Based On Inductive Graph Neural Networks [2.4012886591705738]
  Closeness Centrality (CC) and Betweenness Centrality (BC) are crucial metrics in network analysis. Their practical implementation on extensive networks remains computationally demanding due to their high time complexity. We propose the CNCA-IGE model, which is an inductive graph encoder-decoder model designed to rank nodes based on specified CC or BC metrics.
  arXiv  Detail & Related papers  (2024-03-08T01:23:12Z)
• LitE-SNN: Designing Lightweight and Efficient Spiking Neural Network through Spatial-Temporal Compressive Network Search and Joint Optimization [48.41286573672824]
  Spiking Neural Networks (SNNs) mimic the information-processing mechanisms of the human brain and are highly energy-efficient. We propose a new approach named LitE-SNN that incorporates both spatial and temporal compression into the automated network design process.
  arXiv  Detail & Related papers  (2024-01-26T05:23:11Z)
• EPIM: Efficient Processing-In-Memory Accelerators based on Epitome [78.79382890789607]
  We introduce the Epitome, a lightweight neural operator offering convolution-like functionality. On the software side, we evaluate epitomes' latency and energy on PIM accelerators. We introduce a PIM-aware layer-wise design method to enhance their hardware efficiency.
  arXiv  Detail & Related papers  (2023-11-12T17:56:39Z)
• Pointer Networks with Q-Learning for Combinatorial Optimization [55.2480439325792]
  We introduce the Pointer Q-Network (PQN), a hybrid neural architecture that integrates model-free Q-value policy approximation with Pointer Networks (Ptr-Nets) Our empirical results demonstrate the efficacy of this approach, also testing the model in unstable environments.
  arXiv  Detail & Related papers  (2023-11-05T12:03:58Z)
• ProtoCaps: A Fast and Non-Iterative Capsule Network Routing Method [6.028175460199198]
  We introduce a novel, non-iterative routing mechanism for Capsule Networks. We harness a shared Capsule subspace, negating the need to project each lower-level Capsule to each higher-level Capsule. Our findings underscore the potential of our proposed methodology in enhancing the operational efficiency and performance of Capsule Networks.
  arXiv  Detail & Related papers  (2023-07-19T12:39:40Z)
• Fast Exploration of the Impact of Precision Reduction on Spiking Neural Networks [63.614519238823206]
  Spiking Neural Networks (SNNs) are a practical choice when the target hardware reaches the edge of computing. We employ an Interval Arithmetic (IA) model to develop an exploration methodology that takes advantage of the capability of such a model to propagate the approximation error.
  arXiv  Detail & Related papers  (2022-11-22T15:08:05Z)
• Towards Efficient Capsule Networks [7.1577508803778045]
  Capsule Networks were introduced to enhance explainability of a model, where each capsule is an explicit representation of an object or its parts. We show how pruning with Capsule Network achieves high generalization with less memory requirements, computational effort, and inference and training time.
  arXiv  Detail & Related papers  (2022-08-19T08:03:25Z)
• Optimizing Tensor Network Contraction Using Reinforcement Learning [86.05566365115729]
  We propose a Reinforcement Learning (RL) approach combined with Graph Neural Networks (GNN) to address the contraction ordering problem. The problem is extremely challenging due to the huge search space, the heavy-tailed reward distribution, and the challenging credit assignment. We show how a carefully implemented RL-agent that uses a GNN as the basic policy construct can address these challenges.
  arXiv  Detail & Related papers  (2022-04-18T21:45:13Z)
• Purification and Entanglement Routing on Quantum Networks [55.41644538483948]
  A quantum network equipped with imperfect channel fidelities and limited memory storage time can distribute entanglement between users. We introduce effectives enabling fast path-finding algorithms for maximizing entanglement shared between two nodes on a quantum network.
  arXiv  Detail & Related papers  (2020-11-23T19:00:01Z)
• FlowCaps: Optical Flow Estimation with Capsule Networks For Action Recognition [22.712379018181398]
  Capsule networks (CapsNets) have recently shown promise to excel in most computer vision tasks. We propose a CapsNet-based architecture, termed FlowCaps, which attempts to achieve better correspondence matching via finer-grained, motion-specific, and more-interpretable encoding.
  arXiv  Detail & Related papers  (2020-11-08T11:35:08Z)
• DESCNet: Developing Efficient Scratchpad Memories for Capsule Network Hardware [12.26801463167931]
  Capsule Networks (CapsNets) have improved the generalization ability, as compared to Deep Neural Networks (DNNs) CapsNets pose significantly high computational and memory requirements, making their energy-efficient inference a challenging task. This paper provides, for the first time, an in-depth analysis to highlight the design and management related challenges for the (on-chip) memories deployed in hardware accelerators executing fast CapsNets inference.
  arXiv  Detail & Related papers  (2020-10-12T14:50:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.