Distributed Inference on Mobile Edge and Cloud: A Data-Cartography based Clustering Approach

• URL: http://arxiv.org/abs/2412.16616v1
• Date: Sat, 21 Dec 2024 13:20:26 GMT
• Title: Distributed Inference on Mobile Edge and Cloud: A Data-Cartography based Clustering Approach
• Authors: Divya Jyoti Bajpai, Manjesh Kumar Hanawal,
• Abstract summary: A distributed inference framework can be deployed on mobile devices, edge devices, and the full DNN on the cloud. We introduce a novel method named our, which employs data cartography to assess sample complexity. Our approach significantly lowers inference costs by more than 43% while maintaining a minimal accuracy drop of less than 0.5% compared to performing all inferences on the cloud.
• Score: 5.402030962296633
• License:
• Abstract: The large size of DNNs poses a significant challenge for deployment on devices with limited resources, such as mobile, edge, and IoT platforms. To address this issue, a distributed inference framework can be utilized. In this framework, a small-scale DNN (initial layers) is deployed on mobile devices, a larger version on edge devices, and the full DNN on the cloud. Samples with low complexity (easy) can be processed on mobile, those with moderate complexity (medium) on edge devices, and high complexity (hard) samples on the cloud. Given that the complexity of each sample is unknown in advance, the crucial question in distributed inference is determining the sample complexity for appropriate DNN processing. We introduce a novel method named \our{}, which leverages the Data Cartography approach initially proposed for enhancing DNN generalization. By employing data cartography, we assess sample complexity. \our{} aims to boost accuracy while considering the offloading costs from mobile to edge/cloud. Our experimental results on GLUE datasets, covering a variety of NLP tasks, indicate that our approach significantly lowers inference costs by more than 43\% while maintaining a minimal accuracy drop of less than 0.5\% compared to performing all inferences on the cloud. The source code is available at https://anonymous.4open.science/r/DIMEC-1B04.

Related papers

• Distributed Inference on Mobile Edge and Cloud: An Early Exit based Clustering Approach [5.402030962296633]
  Deep Neural Networks (DNNs) have demonstrated outstanding performance across various domains. A distributed inference setup can be used where a small-sized DNN can be deployed on mobile, a bigger version on the edge, and the full-fledged, on the cloud. We develop a novel approach that utilizes Early Exit (EE) strategies developed to minimize inference latency in DNNs.
  arXiv  Detail & Related papers  (2024-10-06T20:14:27Z)
• MatchNAS: Optimizing Edge AI in Sparse-Label Data Contexts via Automating Deep Neural Network Porting for Mobile Deployment [54.77943671991863]
  MatchNAS is a novel scheme for porting Deep Neural Networks to mobile devices. We optimise a large network family using both labelled and unlabelled data. We then automatically search for tailored networks for different hardware platforms.
  arXiv  Detail & Related papers  (2024-02-21T04:43:12Z)
• Streaming Video Analytics On The Edge With Asynchronous Cloud Support [2.7456483236562437]
  We propose a novel edge-cloud fusion algorithm that fuses edge and cloud predictions, achieving low latency and high accuracy. We focus on object detection in videos (applicable in many video analytics scenarios) and show that the fused edge-cloud predictions can outperform the accuracy of edge-only and cloud-only scenarios by as much as 50%.
  arXiv  Detail & Related papers  (2022-10-04T06:22:13Z)
• CloudAttention: Efficient Multi-Scale Attention Scheme For 3D Point Cloud Learning [81.85951026033787]
  We set transformers in this work and incorporate them into a hierarchical framework for shape classification and part and scene segmentation. We also compute efficient and dynamic global cross attentions by leveraging sampling and grouping at each iteration. The proposed hierarchical model achieves state-of-the-art shape classification in mean accuracy and yields results on par with the previous segmentation methods.
  arXiv  Detail & Related papers  (2022-07-31T21:39:15Z)
• An Adaptive Device-Edge Co-Inference Framework Based on Soft Actor-Critic [72.35307086274912]
  High-dimension parameter model and large-scale mathematical calculation restrict execution efficiency, especially for Internet of Things (IoT) devices. We propose a new Deep Reinforcement Learning (DRL)-Soft Actor Critic for discrete (SAC-d), which generates the emphexit point, emphexit point, and emphcompressing bits by soft policy iterations. Based on the latency and accuracy aware reward design, such an computation can well adapt to the complex environment like dynamic wireless channel and arbitrary processing, and is capable of supporting the 5G URL
  arXiv  Detail & Related papers  (2022-01-09T09:31:50Z)
• Complexity-aware Adaptive Training and Inference for Edge-Cloud Distributed AI Systems [9.273593723275544]
  IoT and machine learning applications create large amounts of data that require real-time processing. We propose a distributed AI system to exploit both the edge and the cloud for training and inference.
  arXiv  Detail & Related papers  (2021-09-14T05:03:54Z)
• Data-Driven Low-Rank Neural Network Compression [8.025818540338518]
  We propose a Data-Driven Low-rank (DDLR) method to reduce the number of parameters of pretrained Deep Neural Networks (DNNs) We show that it is possible to significantly reduce the number of parameters with only a small reduction in classification accuracy.
  arXiv  Detail & Related papers  (2021-07-13T00:10:21Z)
• Learning Semantic Segmentation of Large-Scale Point Clouds with Random Sampling [52.464516118826765]
  We introduce RandLA-Net, an efficient and lightweight neural architecture to infer per-point semantics for large-scale point clouds. The key to our approach is to use random point sampling instead of more complex point selection approaches. Our RandLA-Net can process 1 million points in a single pass up to 200x faster than existing approaches.
  arXiv  Detail & Related papers  (2021-07-06T05:08:34Z)
• Coded Stochastic ADMM for Decentralized Consensus Optimization with Edge Computing [113.52575069030192]
  Big data, including applications with high security requirements, are often collected and stored on multiple heterogeneous devices, such as mobile devices, drones and vehicles. Due to the limitations of communication costs and security requirements, it is of paramount importance to extract information in a decentralized manner instead of aggregating data to a fusion center. We consider the problem of learning model parameters in a multi-agent system with data locally processed via distributed edge nodes. A class of mini-batch alternating direction method of multipliers (ADMM) algorithms is explored to develop the distributed learning model.
  arXiv  Detail & Related papers  (2020-10-02T10:41:59Z)
• Contextual-Bandit Anomaly Detection for IoT Data in Distributed Hierarchical Edge Computing [65.78881372074983]
  IoT devices can hardly afford complex deep neural networks (DNN) models, and offloading anomaly detection tasks to the cloud incurs long delay. We propose and build a demo for an adaptive anomaly detection approach for distributed hierarchical edge computing (HEC) systems. We show that our proposed approach significantly reduces detection delay without sacrificing accuracy, as compared to offloading detection tasks to the cloud.
  arXiv  Detail & Related papers  (2020-04-15T06:13:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.