Conditionally Deep Hybrid Neural Networks Across Edge and Cloud

• URL: http://arxiv.org/abs/2005.10851v1
• Date: Thu, 21 May 2020 18:18:43 GMT
• Title: Conditionally Deep Hybrid Neural Networks Across Edge and Cloud
• Authors: Yinghan Long, Indranil Chakraborty, Kaushik Roy
• Abstract summary: We propose a conditionally deep hybrid neural network for enabling AI-based fog computing. The proposed network can be deployed in a distributed manner, consisting of quantized layers and early exits at the edge and full-precision layers on the cloud. We show that with binarized layers at the edge, the proposed conditional hybrid network can process 65% of inferences at the edge, leading to 5.5x computational energy reduction.
• Score: 6.442928757682793
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The pervasiveness of "Internet-of-Things" in our daily life has led to a recent surge in fog computing, encompassing a collaboration of cloud computing and edge intelligence. To that effect, deep learning has been a major driving force towards enabling such intelligent systems. However, growing model sizes in deep learning pose a significant challenge towards deployment in resource-constrained edge devices. Moreover, in a distributed intelligence environment, efficient workload distribution is necessary between edge and cloud systems. To address these challenges, we propose a conditionally deep hybrid neural network for enabling AI-based fog computing. The proposed network can be deployed in a distributed manner, consisting of quantized layers and early exits at the edge and full-precision layers on the cloud. During inference, if an early exit has high confidence in the classification results, it would allow samples to exit at the edge, and the deeper layers on the cloud are activated conditionally, which can lead to improved energy efficiency and inference latency. We perform an extensive design space exploration with the goal of minimizing energy consumption at the edge while achieving state-of-the-art classification accuracies on image classification tasks. We show that with binarized layers at the edge, the proposed conditional hybrid network can process 65% of inferences at the edge, leading to 5.5x computational energy reduction with minimal accuracy degradation on CIFAR-10 dataset. For the more complex dataset CIFAR-100, we observe that the proposed network with 4-bit quantization at the edge achieves 52% early classification at the edge with 4.8x energy reduction. The analysis gives us insights on designing efficient hybrid networks which achieve significantly higher energy efficiency than full-precision networks for edge-cloud based distributed intelligence systems.

Related papers

• Growing Efficient Accurate and Robust Neural Networks on the Edge [0.9208007322096533]
  Current solutions rely on the Cloud to train and compress models before deploying to the Edge. This incurs high energy and latency costs in transmitting locally acquired field data to the Cloud while also raising privacy concerns. We propose GEARnn to grow and train robust networks entirely on the Edge device.
  arXiv  Detail & Related papers  (2024-10-10T08:01:42Z)
• I-SplitEE: Image classification in Split Computing DNNs with Early Exits [5.402030962296633]
  Large size of Deep Neural Networks (DNNs) hinders deploying them on resource-constrained devices like edge, mobile, and IoT platforms. Our work presents an innovative unified approach merging early exits and split computing. I-SplitEE is an online unsupervised algorithm ideal for scenarios lacking ground truths and with sequential data.
  arXiv  Detail & Related papers  (2024-01-19T07:44:32Z)
• Artificial Intelligence Empowered Multiple Access for Ultra Reliable and Low Latency THz Wireless Networks [76.89730672544216]
  Terahertz (THz) wireless networks are expected to catalyze the beyond fifth generation (B5G) era. To satisfy the ultra-reliability and low-latency demands of several B5G applications, novel mobility management approaches are required. This article presents a holistic MAC layer approach that enables intelligent user association and resource allocation, as well as flexible and adaptive mobility management.
  arXiv  Detail & Related papers  (2022-08-17T03:00:24Z)
• Edge YOLO: Real-Time Intelligent Object Detection System Based on Edge-Cloud Cooperation in Autonomous Vehicles [5.295478084029605]
  We propose an object detection (OD) system based on edge-cloud cooperation and reconstructive convolutional neural networks. This system can effectively avoid the excessive dependence on computing power and uneven distribution of cloud computing resources. We experimentally demonstrate the reliability and efficiency of Edge YOLO on COCO 2017 and KITTI data sets.
  arXiv  Detail & Related papers  (2022-05-30T09:16:35Z)
• POEM: 1-bit Point-wise Operations based on Expectation-Maximization for Efficient Point Cloud Processing [53.74076015905961]
  We introduce point-wise operations based on Expectation-Maximization into BNNs for efficient point cloud processing. Our POEM surpasses existing the state-of-the-art binary point cloud networks by a significant margin, up to 6.7 %.
  arXiv  Detail & Related papers  (2021-11-26T09:45:01Z)
• Deep Reinforcement Learning Based Multidimensional Resource Management for Energy Harvesting Cognitive NOMA Communications [64.1076645382049]
  Combination of energy harvesting (EH), cognitive radio (CR), and non-orthogonal multiple access (NOMA) is a promising solution to improve energy efficiency. In this paper, we study the spectrum, energy, and time resource management for deterministic-CR-NOMA IoT systems.
  arXiv  Detail & Related papers  (2021-09-17T08:55:48Z)
• Semi-supervised Network Embedding with Differentiable Deep Quantisation [81.49184987430333]
  We develop d-SNEQ, a differentiable quantisation method for network embedding. d-SNEQ incorporates a rank loss to equip the learned quantisation codes with rich high-order information. It is able to substantially compress the size of trained embeddings, thus reducing storage footprint and accelerating retrieval speed.
  arXiv  Detail & Related papers  (2021-08-20T11:53:05Z)
• On Topology Optimization and Routing in Integrated Access and Backhaul Networks: A Genetic Algorithm-based Approach [70.85399600288737]
  We study the problem of topology optimization and routing in IAB networks. We develop efficient genetic algorithm-based schemes for both IAB node placement and non-IAB backhaul link distribution. We discuss the main challenges for enabling mesh-based IAB networks.
  arXiv  Detail & Related papers  (2021-02-14T21:52:05Z)
• Towards AIOps in Edge Computing Environments [60.27785717687999]
  This paper describes the system design of an AIOps platform which is applicable in heterogeneous, distributed environments. It is feasible to collect metrics with a high frequency and simultaneously run specific anomaly detection algorithms directly on edge devices.
  arXiv  Detail & Related papers  (2021-02-12T09:33:00Z)
• CoEdge: Cooperative DNN Inference with Adaptive Workload Partitioning over Heterogeneous Edge Devices [39.09319776243573]
  CoEdge is a distributed Deep Neural Network (DNN) computing system that orchestrates cooperative inference over heterogeneous edge devices. CoEdge saves energy with close inference latency, achieving up to 25.5%66.9% energy reduction for four widely-adopted CNN models.
  arXiv  Detail & Related papers  (2020-12-06T13:15:52Z)
• Deep Reinforcement Learning for Adaptive Network Slicing in 5G for Intelligent Vehicular Systems and Smart Cities [19.723551683930776]
  We develop a network slicing model based on a cluster of fog nodes (FNs) coordinated with an edge controller (EC) For each service request in a cluster, the EC decides which FN to execute the task, locally serve the request at the edge, or to reject the task and refer it to the cloud. We propose a deep reinforcement learning (DRL) solution to adaptively learn the optimal slicing policy.
  arXiv  Detail & Related papers  (2020-10-19T23:30:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.