A Sequential Concept Drift Detection Method for On-Device Learning on Low-End Edge Devices

• URL: http://arxiv.org/abs/2212.09637v1
• Date: Mon, 19 Dec 2022 17:13:59 GMT
• Title: A Sequential Concept Drift Detection Method for On-Device Learning on Low-End Edge Devices
• Authors: Takeya Yamada, Hiroki Matsutani
• Abstract summary: A practical issue of edge AI systems is that data distributions of trained dataset and deployed environment may differ due to noise and environmental changes over time. We propose a lightweight concept drift detection method in cooperation with a recently proposed on-device learning technique of neural networks.
• Score: 2.520804666686246
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A practical issue of edge AI systems is that data distributions of trained dataset and deployed environment may differ due to noise and environmental changes over time. Such a phenomenon is known as a concept drift, and this gap degrades the performance of edge AI systems and may introduce system failures. To address this gap, a retraining of neural network models triggered by concept drift detection is a practical approach. However, since available compute resources are strictly limited in edge devices, in this paper we propose a lightweight concept drift detection method in cooperation with a recently proposed on-device learning technique of neural networks. In this case, both the neural network retraining and the proposed concept drift detection are done by sequential computation only to reduce computation cost and memory utilization. Evaluation results of the proposed approach shows that while the accuracy is decreased by 3.8%-4.3% compared to existing batch-based detection methods, it decreases the memory size by 88.9%-96.4% and the execution time by 1.3%-83.8%. As a result, the combination of the neural network retraining and the proposed concept drift detection method is demonstrated on Raspberry Pi Pico that has 264kB memory.

Related papers

• SEF: A Method for Computing Prediction Intervals by Shifting the Error Function in Neural Networks [0.0]
  SEF (Shifting the Error Function) method presented in this paper is a new method that belongs to this category of methods. The proposed approach involves training a single neural network three times, thus generating an estimate along with the corresponding upper and lower bounds for a given problem. This innovative process effectively produces PIs, resulting in a robust and efficient technique for uncertainty quantification.
  arXiv  Detail & Related papers  (2024-09-08T19:46:45Z)
• Deep Neural Networks Tend To Extrapolate Predictably [51.303814412294514]
  neural network predictions tend to be unpredictable and overconfident when faced with out-of-distribution (OOD) inputs. We observe that neural network predictions often tend towards a constant value as input data becomes increasingly OOD. We show how one can leverage our insights in practice to enable risk-sensitive decision-making in the presence of OOD inputs.
  arXiv  Detail & Related papers  (2023-10-02T03:25:32Z)
• The Cascaded Forward Algorithm for Neural Network Training [61.06444586991505]
  We propose a new learning framework for neural networks, namely Cascaded Forward (CaFo) algorithm, which does not rely on BP optimization as that in FF. Unlike FF, our framework directly outputs label distributions at each cascaded block, which does not require generation of additional negative samples. In our framework each block can be trained independently, so it can be easily deployed into parallel acceleration systems.
  arXiv  Detail & Related papers  (2023-03-17T02:01:11Z)
• Estimation of Sea State Parameters from Ship Motion Responses Using Attention-based Neural Networks [0.6193838300896448]
  We apply the novel attention-based neural network (AT-NN) for estimating sea state parameters from raw time-series data of ship pitch, heave, and roll motions. It has been successfully demonstrated that the proposed approaches by modified state-of-the-art techniques reduced estimation MSE by 23% and MAE by 16% compared to the original methods.
  arXiv  Detail & Related papers  (2023-01-21T13:21:50Z)
• Pushing the Limits of Asynchronous Graph-based Object Detection with Event Cameras [62.70541164894224]
  We introduce several architecture choices which allow us to scale the depth and complexity of such models while maintaining low computation. Our method runs 3.7 times faster than a dense graph neural network, taking only 8.4 ms per forward pass.
  arXiv  Detail & Related papers  (2022-11-22T15:14:20Z)
• Fast Exploration of the Impact of Precision Reduction on Spiking Neural Networks [63.614519238823206]
  Spiking Neural Networks (SNNs) are a practical choice when the target hardware reaches the edge of computing. We employ an Interval Arithmetic (IA) model to develop an exploration methodology that takes advantage of the capability of such a model to propagate the approximation error.
  arXiv  Detail & Related papers  (2022-11-22T15:08:05Z)
• Drift Reduction for Monocular Visual Odometry of Intelligent Vehicles using Feedforward Neural Networks [0.0]
  A visual odometry algorithm computes the incremental motion of the vehicle between the successive camera frames. The proposed neural network reduces the errors in the pose estimation of the vehicle. The results show the efficacy of the proposed approach in reducing the errors in the incremental orientation estimation.
  arXiv  Detail & Related papers  (2022-07-02T21:28:41Z)
• Autoregressive based Drift Detection Method [0.0]
  We propose a new concept drift detection method based on autoregressive models called ADDM. Our results show that this new concept drift detection method outperforms the state-of-the-art drift detection methods.
  arXiv  Detail & Related papers  (2022-03-09T14:36:16Z)
• Detecting Concept Drift With Neural Network Model Uncertainty [0.0]
  Uncertainty Drift Detection (UDD) is able to detect drifts without access to true labels. In contrast to input data-based drift detection, our approach considers the effects of the current input data on the properties of the prediction model. We show that UDD outperforms other state-of-the-art strategies on two synthetic as well as ten real-world data sets for both regression and classification tasks.
  arXiv  Detail & Related papers  (2021-07-05T08:56:36Z)
• SignalNet: A Low Resolution Sinusoid Decomposition and Estimation Network [79.04274563889548]
  We propose SignalNet, a neural network architecture that detects the number of sinusoids and estimates their parameters from quantized in-phase and quadrature samples. We introduce a worst-case learning threshold for comparing the results of our network relative to the underlying data distributions. In simulation, we find that our algorithm is always able to surpass the threshold for three-bit data but often cannot exceed the threshold for one-bit data.
  arXiv  Detail & Related papers  (2021-06-10T04:21:20Z)
• Identity-Aware Attribute Recognition via Real-Time Distributed Inference in Mobile Edge Clouds [53.07042574352251]
  We design novel models for pedestrian attribute recognition with re-ID in an MEC-enabled camera monitoring system. We propose a novel inference framework with a set of distributed modules, by jointly considering the attribute recognition and person re-ID. We then devise a learning-based algorithm for the distributions of the modules of the proposed distributed inference framework.
  arXiv  Detail & Related papers  (2020-08-12T12:03:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.