Fault-Tolerant Collaborative Inference through the Edge-PRUNE Framework
- URL: http://arxiv.org/abs/2206.08152v1
- Date: Thu, 16 Jun 2022 13:16:53 GMT
- Title: Fault-Tolerant Collaborative Inference through the Edge-PRUNE Framework
- Authors: Jani Boutellier, Bo Tan, Jari Nurmi
- Abstract summary: Collaborative inference is a vehicle for distributing computation load, reducing latency, and addressing privacy preservation in communications.
This paper presents the Edge-PRUNE distributed computing framework, built on a formally defined model of computation, which provides a flexible infrastructure for fault tolerant collaborative inference.
- Score: 4.984601297028258
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Collaborative inference has received significant research interest in machine
learning as a vehicle for distributing computation load, reducing latency, as
well as addressing privacy preservation in communications. Recent collaborative
inference frameworks have adopted dynamic inference methodologies such as
early-exit and run-time partitioning of neural networks. However, as machine
learning frameworks scale in the number of inference inputs, e.g., in
surveillance applications, fault tolerance related to device failure needs to
be considered. This paper presents the Edge-PRUNE distributed computing
framework, built on a formally defined model of computation, which provides a
flexible infrastructure for fault tolerant collaborative inference. The
experimental section of this work shows results on achievable inference time
savings by collaborative inference, presents fault tolerant system topologies
and analyzes their cost in terms of execution time overhead.
Related papers
- Failure-Resilient Distributed Inference with Model Compression over Heterogeneous Edge Devices [9.423705897088672]
We present RoCoIn, a robust cooperative inference mechanism for locally distributed execution of deep neural network-based inference tasks over heterogeneous edge devices.
It creates a set of independent and compact student models that are learned from a large model using knowledge distillation for distributed deployment.
In particular, the devices are strategically grouped to redundantly deploy and execute the same student model such that the inference process is resilient to any local failures.
arXiv Detail & Related papers (2024-06-20T10:43:53Z) - Graph Machine Learning based Doubly Robust Estimator for Network Causal Effects [17.44202934049009]
We propose a novel methodology that combines graph machine learning approaches with the double machine learning framework.
We demonstrate our method is accurate, robust, and scalable via an extensive simulation study.
arXiv Detail & Related papers (2024-03-17T20:23:42Z) - Scalable Federated Unlearning via Isolated and Coded Sharding [76.12847512410767]
Federated unlearning has emerged as a promising paradigm to erase the client-level data effect.
This paper proposes a scalable federated unlearning framework based on isolated sharding and coded computing.
arXiv Detail & Related papers (2024-01-29T08:41:45Z) - Advancing Counterfactual Inference through Nonlinear Quantile Regression [77.28323341329461]
We propose a framework for efficient and effective counterfactual inference implemented with neural networks.
The proposed approach enhances the capacity to generalize estimated counterfactual outcomes to unseen data.
Empirical results conducted on multiple datasets offer compelling support for our theoretical assertions.
arXiv Detail & Related papers (2023-06-09T08:30:51Z) - Matched Machine Learning: A Generalized Framework for Treatment Effect
Inference With Learned Metrics [87.05961347040237]
We introduce Matched Machine Learning, a framework that combines the flexibility of machine learning black boxes with the interpretability of matching.
Our framework uses machine learning to learn an optimal metric for matching units and estimating outcomes.
We show empirically that instances of Matched Machine Learning perform on par with black-box machine learning methods and better than existing matching methods for similar problems.
arXiv Detail & Related papers (2023-04-03T19:32:30Z) - Design and Prototyping Distributed CNN Inference Acceleration in Edge
Computing [85.74517957717363]
HALP accelerates inference by designing a seamless collaboration among edge devices (EDs) in Edge Computing.
Experiments show that the distributed inference HALP achieves 1.7x inference acceleration for VGG-16.
It is shown that the model selection with distributed inference HALP can significantly improve service reliability.
arXiv Detail & Related papers (2022-11-24T19:48:30Z) - Finite-Time Consensus Learning for Decentralized Optimization with
Nonlinear Gossiping [77.53019031244908]
We present a novel decentralized learning framework based on nonlinear gossiping (NGO), that enjoys an appealing finite-time consensus property to achieve better synchronization.
Our analysis on how communication delay and randomized chats affect learning further enables the derivation of practical variants.
arXiv Detail & Related papers (2021-11-04T15:36:25Z) - Communication-Computation Efficient Device-Edge Co-Inference via AutoML [4.06604174802643]
Device-edge co-inference partitions a deep neural network between a resource-constrained mobile device and an edge server.
On-device model sparsity level and intermediate feature compression ratio have direct impacts on workload and communication overhead.
We propose a novel automated machine learning (AutoML) framework based on deep reinforcement learning (DRL)
arXiv Detail & Related papers (2021-08-30T06:36:30Z) - Estimating the Effects of Continuous-valued Interventions using
Generative Adversarial Networks [103.14809802212535]
We build on the generative adversarial networks (GANs) framework to address the problem of estimating the effect of continuous-valued interventions.
Our model, SCIGAN, is flexible and capable of simultaneously estimating counterfactual outcomes for several different continuous interventions.
To address the challenges presented by shifting to continuous interventions, we propose a novel architecture for our discriminator.
arXiv Detail & Related papers (2020-02-27T18:46:21Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.