Tiny Machine Learning for Concept Drift
- URL: http://arxiv.org/abs/2107.14759v1
- Date: Fri, 30 Jul 2021 17:02:04 GMT
- Title: Tiny Machine Learning for Concept Drift
- Authors: Simone Disabato and Manuel Roveri
- Abstract summary: This paper introduces a Tiny Machine Learning for Concept Drift (TML-CD) solution based on deep learning feature extractors and a k-nearest neighbors.
The adaptation module continuously updates the knowledge base of TML-CD to deal with concept drift affecting the data-generating process.
The porting of TML-CD on three off-the-shelf micro-controller units shows the feasibility of what is proposed in real-world pervasive systems.
- Score: 8.452237741722726
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Tiny Machine Learning (TML) is a new research area whose goal is to design
machine and deep learning techniques able to operate in Embedded Systems and
IoT units, hence satisfying the severe technological constraints on memory,
computation, and energy characterizing these pervasive devices. Interestingly,
the related literature mainly focused on reducing the computational and memory
demand of the inference phase of machine and deep learning models. At the same
time, the training is typically assumed to be carried out in Cloud or edge
computing systems (due to the larger memory and computational requirements).
This assumption results in TML solutions that might become obsolete when the
process generating the data is affected by concept drift (e.g., due to
periodicity or seasonality effect, faults or malfunctioning affecting sensors
or actuators, or changes in the users' behavior), a common situation in
real-world application scenarios. For the first time in the literature, this
paper introduces a Tiny Machine Learning for Concept Drift (TML-CD) solution
based on deep learning feature extractors and a k-nearest neighbors classifier
integrating a hybrid adaptation module able to deal with concept drift
affecting the data-generating process. This adaptation module continuously
updates (in a passive way) the knowledge base of TML-CD and, at the same time,
employs a Change Detection Test to inspect for changes (in an active way) to
quickly adapt to concept drift by removing the obsolete knowledge. Experimental
results on both image and audio benchmarks show the effectiveness of the
proposed solution, whilst the porting of TML-CD on three off-the-shelf
micro-controller units shows the feasibility of what is proposed in real-world
pervasive systems.
Related papers
- A Continual and Incremental Learning Approach for TinyML On-device Training Using Dataset Distillation and Model Size Adaption [0.4345992906143838]
A new algorithm for incremental learning in the context of Tiny Machine learning (TinyML) is presented.
It is optimized for low-performance and energy efficient embedded devices.
Results show that the proposed algorithm offers a promising approach for TinyML incremental learning on embedded devices.
arXiv Detail & Related papers (2024-09-11T09:02:33Z) - Learn while Unlearn: An Iterative Unlearning Framework for Generative Language Models [49.043599241803825]
Iterative Contrastive Unlearning (ICU) framework consists of three core components.
A Knowledge Unlearning Induction module removes specific knowledge through an unlearning loss.
A Contrastive Learning Enhancement module to preserve the model's expressive capabilities against the pure unlearning goal.
And an Iterative Unlearning Refinement module that dynamically assess the unlearning extent on specific data pieces and make iterative update.
arXiv Detail & Related papers (2024-07-25T07:09:35Z) - On-device Online Learning and Semantic Management of TinyML Systems [8.183732025472766]
This study aims to bridge the gap between prototyping single TinyML models and developing reliable TinyML systems in production.
We propose online learning to enable training on constrained devices, adapting local models towards the latest field conditions.
We present semantic management for the joint management of models and devices at scale.
arXiv Detail & Related papers (2024-05-13T10:03:34Z) - Goldfish: An Efficient Federated Unlearning Framework [3.956103498302838]
Goldfish is a new framework for machine unlearning algorithms.
It comprises four modules: basic model, loss function, optimization, and extension.
To address the challenge of low validity in existing machine unlearning algorithms, we propose a novel loss function.
arXiv Detail & Related papers (2024-04-04T03:29:41Z) - Random resistive memory-based deep extreme point learning machine for
unified visual processing [67.51600474104171]
We propose a novel hardware-software co-design, random resistive memory-based deep extreme point learning machine (DEPLM)
Our co-design system achieves huge energy efficiency improvements and training cost reduction when compared to conventional systems.
arXiv Detail & Related papers (2023-12-14T09:46:16Z) - Incremental Online Learning Algorithms Comparison for Gesture and Visual
Smart Sensors [68.8204255655161]
This paper compares four state-of-the-art algorithms in two real applications: gesture recognition based on accelerometer data and image classification.
Our results confirm these systems' reliability and the feasibility of deploying them in tiny-memory MCUs.
arXiv Detail & Related papers (2022-09-01T17:05:20Z) - SOLIS -- The MLOps journey from data acquisition to actionable insights [62.997667081978825]
In this paper we present a unified deployment pipeline and freedom-to-operate approach that supports all requirements while using basic cross-platform tensor framework and script language engines.
This approach however does not supply the needed procedures and pipelines for the actual deployment of machine learning capabilities in real production grade systems.
arXiv Detail & Related papers (2021-12-22T14:45:37Z) - Model of the Weak Reset Process in HfOx Resistive Memory for Deep
Learning Frameworks [0.6745502291821955]
We present a model of the weak RESET process in hafnium oxide RRAM.
We integrate this model within the PyTorch deep learning framework.
We use this tool to train Binarized Neural Networks for the MNIST handwritten digit recognition task.
arXiv Detail & Related papers (2021-07-02T08:50:35Z) - Automated Machine Learning Techniques for Data Streams [91.3755431537592]
This paper surveys the state-of-the-art open-source AutoML tools, applies them to data collected from streams, and measures how their performance changes over time.
The results show that off-the-shelf AutoML tools can provide satisfactory results but in the presence of concept drift, detection or adaptation techniques have to be applied to maintain the predictive accuracy over time.
arXiv Detail & Related papers (2021-06-14T11:42:46Z) - An Efficient One-Class SVM for Anomaly Detection in the Internet of
Things [25.78558553080511]
Insecure Internet of things (IoT) devices pose significant threats to critical infrastructure and the Internet at large.
detecting anomalous behavior from these devices remains of critical importance.
One-Class Support Vector Machines (OCSVM) are one of the state-of-the-art approaches for novelty detection.
arXiv Detail & Related papers (2021-04-22T15:59:56Z) - Transfer Learning without Knowing: Reprogramming Black-box Machine
Learning Models with Scarce Data and Limited Resources [78.72922528736011]
We propose a novel approach, black-box adversarial reprogramming (BAR), that repurposes a well-trained black-box machine learning model.
Using zeroth order optimization and multi-label mapping techniques, BAR can reprogram a black-box ML model solely based on its input-output responses.
BAR outperforms state-of-the-art methods and yields comparable performance to the vanilla adversarial reprogramming method.
arXiv Detail & Related papers (2020-07-17T01:52:34Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.