Prune2Edge: A Multi-Phase Pruning Pipelines to Deep Ensemble Learning in
IIoT
- URL: http://arxiv.org/abs/2004.04710v2
- Date: Thu, 21 Jan 2021 16:05:23 GMT
- Title: Prune2Edge: A Multi-Phase Pruning Pipelines to Deep Ensemble Learning in
IIoT
- Authors: Besher Alhalabi, Mohamed Gaber, Shadi Basurra
- Abstract summary: We propose a novel edge-based multi-phase pruning pipelines to ensemble learning on IIoT devices.
In the first phase, we generate a diverse ensemble of pruned models, then we apply integer quantisation, next we prune the generated ensemble using a clustering-based technique.
Our proposed approach was able to outperform the predictability levels of a baseline model.
- Score: 0.0
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Most recently, with the proliferation of IoT devices, computational nodes in
manufacturing systems IIoT(Industrial-Internet-of-things) and the lunch of 5G
networks, there will be millions of connected devices generating a massive
amount of data. In such an environment, the controlling systems need to be
intelligent enough to deal with a vast amount of data to detect defects in a
real-time process. Driven by such a need, artificial intelligence models such
as deep learning have to be deployed into IIoT systems. However, learning and
using deep learning models are computationally expensive, so an IoT device with
limited computational power could not run such models. To tackle this issue,
edge intelligence had emerged as a new paradigm towards running Artificial
Intelligence models on edge devices. Although a considerable amount of studies
have been proposed in this area, the research is still in the early stages. In
this paper, we propose a novel edge-based multi-phase pruning pipelines to
ensemble learning on IIoT devices. In the first phase, we generate a diverse
ensemble of pruned models, then we apply integer quantisation, next we prune
the generated ensemble using a clustering-based technique. Finally, we choose
the best representative from each generated cluster to be deployed to a
distributed IoT environment. On CIFAR-100 and CIFAR-10, our proposed approach
was able to outperform the predictability levels of a baseline model (up to
7%), more importantly, the generated learners have small sizes (up to 90%
reduction in the model size) that minimise the required computational
capabilities to make an inference on the resource-constraint devices.
Related papers
- Efficient Intrusion Detection: Combining $χ^2$ Feature Selection with CNN-BiLSTM on the UNSW-NB15 Dataset [2.239394800147746]
Intrusion Detection Systems (IDSs) have played a significant role in the detection and prevention of cyber-attacks in traditional computing systems.
The limited computational resources available on Internet of Things (IoT) devices pose a challenge for deploying conventional computing-based IDSs.
We present an effective IDS model that addresses this issue by combining a lightweight Convolutional Neural Network (CNN) with bidirectional Long Short-Term Memory (BiLSTM)
arXiv Detail & Related papers (2024-07-20T17:41:16Z) - Advancing IIoT with Over-the-Air Federated Learning: The Role of Iterative Magnitude Pruning [14.818439341517733]
Industrial Internet of Things (IIoT) under Industry 4.0 heralds an era of interconnected smart devices.
federated learning (FL) addresses data privacy and security among devices.
FL enables edge sensors to learn and adapt using their data locally, without explicit sharing of confidential data.
arXiv Detail & Related papers (2024-03-21T04:15:56Z) - Distributed Inference and Fine-tuning of Large Language Models Over The
Internet [91.00270820533272]
Large language models (LLMs) are useful in many NLP tasks and become more capable with size.
These models require high-end hardware, making them inaccessible to most researchers.
We develop fault-tolerant inference algorithms and load-balancing protocols that automatically assign devices to maximize the total system throughput.
arXiv Detail & Related papers (2023-12-13T18:52:49Z) - Computational Intelligence and Deep Learning for Next-Generation
Edge-Enabled Industrial IoT [51.68933585002123]
We investigate how to deploy computational intelligence and deep learning (DL) in edge-enabled industrial IoT networks.
In this paper, we propose a novel multi-exit-based federated edge learning (ME-FEEL) framework.
In particular, the proposed ME-FEEL can achieve an accuracy gain up to 32.7% in the industrial IoT networks with the severely limited resources.
arXiv Detail & Related papers (2021-10-28T08:14:57Z) - LCS: Learning Compressible Subspaces for Adaptive Network Compression at
Inference Time [57.52251547365967]
We propose a method for training a "compressible subspace" of neural networks that contains a fine-grained spectrum of models.
We present results for achieving arbitrarily fine-grained accuracy-efficiency trade-offs at inference time for structured and unstructured sparsity.
Our algorithm extends to quantization at variable bit widths, achieving accuracy on par with individually trained networks.
arXiv Detail & Related papers (2021-10-08T17:03:34Z) - Adaptive Anomaly Detection for Internet of Things in Hierarchical Edge
Computing: A Contextual-Bandit Approach [81.5261621619557]
We propose an adaptive anomaly detection scheme with hierarchical edge computing (HEC)
We first construct multiple anomaly detection DNN models with increasing complexity, and associate each of them to a corresponding HEC layer.
Then, we design an adaptive model selection scheme that is formulated as a contextual-bandit problem and solved by using a reinforcement learning policy network.
arXiv Detail & Related papers (2021-08-09T08:45:47Z) - ESAI: Efficient Split Artificial Intelligence via Early Exiting Using
Neural Architecture Search [6.316693022958222]
Deep neural networks have been outperforming conventional machine learning algorithms in many computer vision-related tasks.
The majority of devices are harnessing the cloud computing methodology in which outstanding deep learning models are responsible for analyzing the data on the server.
In this paper, a new framework for deploying on IoT devices has been proposed which can take advantage of both the cloud and the on-device models.
arXiv Detail & Related papers (2021-06-21T04:47:53Z) - IoT Security: Botnet detection in IoT using Machine learning [0.0]
This research work is to propose an innovative model using machine learning algorithm to detect and mitigate botnet-based distributed denial of service (DDoS) attack in IoT network.
Our proposed model tackles the security issue concerning the threats from bots.
arXiv Detail & Related papers (2021-04-06T01:47:50Z) - Coded Stochastic ADMM for Decentralized Consensus Optimization with Edge
Computing [113.52575069030192]
Big data, including applications with high security requirements, are often collected and stored on multiple heterogeneous devices, such as mobile devices, drones and vehicles.
Due to the limitations of communication costs and security requirements, it is of paramount importance to extract information in a decentralized manner instead of aggregating data to a fusion center.
We consider the problem of learning model parameters in a multi-agent system with data locally processed via distributed edge nodes.
A class of mini-batch alternating direction method of multipliers (ADMM) algorithms is explored to develop the distributed learning model.
arXiv Detail & Related papers (2020-10-02T10:41:59Z) - Deep Learning for Ultra-Reliable and Low-Latency Communications in 6G
Networks [84.2155885234293]
We first summarize how to apply data-driven supervised deep learning and deep reinforcement learning in URLLC.
To address these open problems, we develop a multi-level architecture that enables device intelligence, edge intelligence, and cloud intelligence for URLLC.
arXiv Detail & Related papers (2020-02-22T14:38:11Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.