Energy-Efficient Adaptive Machine Learning on IoT End-Nodes With Class-Dependent Confidence

• URL: http://arxiv.org/abs/2204.03431v1
• Date: Thu, 7 Apr 2022 13:22:52 GMT
• Title: Energy-Efficient Adaptive Machine Learning on IoT End-Nodes With Class-Dependent Confidence
• Authors: Francesco Daghero, Alessio Burrello, Daniele Jahier Pagliari, Luca Benini, Enrico Macii, Massimo Poncino
• Abstract summary: An effective way to obtain energy-efficiency with small accuracy drops is to sequentially execute a set of increasingly complex models. Current methods employ a single threshold on the output probabilities produced by each model. We show that our method can significantly reduce the energy consumption compared to the single-threshold approach.
• Score: 22.225875583595027
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Energy-efficient machine learning models that can run directly on edge devices are of great interest in IoT applications, as they can reduce network pressure and response latency, and improve privacy. An effective way to obtain energy-efficiency with small accuracy drops is to sequentially execute a set of increasingly complex models, early-stopping the procedure for "easy" inputs that can be confidently classified by the smallest models. As a stopping criterion, current methods employ a single threshold on the output probabilities produced by each model. In this work, we show that such a criterion is sub-optimal for datasets that include classes of different complexity, and we demonstrate a more general approach based on per-classes thresholds. With experiments on a low-power end-node, we show that our method can significantly reduce the energy consumption compared to the single-threshold approach.

Related papers

• A Green Multi-Attribute Client Selection for Over-The-Air Federated Learning: A Grey-Wolf-Optimizer Approach [5.277822313069301]
  Over-the-air (OTA) FL was introduced to tackle these challenges by disseminating model updates without direct device-to-device connections or centralized servers. OTA-FL brought forth limitations associated with heightened energy consumption and network latency. We propose a multi-attribute client selection framework employing the grey wolf (GWO) to strategically control the number of participants in each round.
  arXiv  Detail & Related papers  (2024-09-16T20:03:57Z)
• Adv-KD: Adversarial Knowledge Distillation for Faster Diffusion Sampling [2.91204440475204]
  Diffusion Probabilistic Models (DPMs) have emerged as a powerful class of deep generative models. They rely on sequential denoising steps during sample generation. We propose a novel method that integrates denoising phases directly into the model's architecture.
  arXiv  Detail & Related papers  (2024-05-31T08:19:44Z)
• Optimal design of experiments in the context of machine-learning inter-atomic potentials: improving the efficiency and transferability of kernel based methods [0.7234862895932991]
  Data-driven, machine learning (ML) models of atomistic interactions can relate nuanced aspects of atomic arrangements into predictions of energies and forces. The main challenge stems from the fact that descriptors of chemical environments are often sparse high-dimensional objects without a well-defined continuous metric. We will demonstrate that classical concepts of statistical planning of experiments and optimal design can help to mitigate such problems at a relatively low computational cost.
  arXiv  Detail & Related papers  (2024-05-14T14:14:23Z)
• Self-Supervised Learning with Generative Adversarial Networks for Electron Microscopy [0.0]
  We show how self-supervised pretraining facilitates efficient fine-tuning for a spectrum of downstream tasks. We demonstrate the versatility of self-supervised pretraining across various downstream tasks in the context of electron microscopy.
  arXiv  Detail & Related papers  (2024-02-28T12:25:01Z)
• Boosting Inference Efficiency: Unleashing the Power of Parameter-Shared Pre-trained Language Models [109.06052781040916]
  We introduce a technique to enhance the inference efficiency of parameter-shared language models. We also propose a simple pre-training technique that leads to fully or partially shared models. Results demonstrate the effectiveness of our methods on both autoregressive and autoencoding PLMs.
  arXiv  Detail & Related papers  (2023-10-19T15:13:58Z)
• Effective Pre-Training Objectives for Transformer-based Autoencoders [97.99741848756302]
  We study trade-offs between efficiency, cost and accuracy of Transformer encoders. We combine features of common objectives and create new effective pre-training approaches.
  arXiv  Detail & Related papers  (2022-10-24T18:39:44Z)
• Semantic Driven Energy based Out-of-Distribution Detection [0.4640835690336652]
  Energy based OOD methods have proved to be promising and achieved impressive performance. We propose semantic driven energy based method, which is an end-to-end trainable system and easy to optimize. We find that, our novel approach enhances outlier detection and achieve state-of-the-art as an energy-based model on common benchmarks.
  arXiv  Detail & Related papers  (2022-08-23T07:40:34Z)
• Your Autoregressive Generative Model Can be Better If You Treat It as an Energy-Based One [83.5162421521224]
  We propose a unique method termed E-ARM for training autoregressive generative models. E-ARM takes advantage of a well-designed energy-based learning objective. We show that E-ARM can be trained efficiently and is capable of alleviating the exposure bias problem.
  arXiv  Detail & Related papers  (2022-06-26T10:58:41Z)
• HyperImpute: Generalized Iterative Imputation with Automatic Model Selection [77.86861638371926]
  We propose a generalized iterative imputation framework for adaptively and automatically configuring column-wise models. We provide a concrete implementation with out-of-the-box learners, simulators, and interfaces.
  arXiv  Detail & Related papers  (2022-06-15T19:10:35Z)
• Online reinforcement learning with sparse rewards through an active inference capsule [62.997667081978825]
  This paper introduces an active inference agent which minimizes the novel free energy of the expected future. Our model is capable of solving sparse-reward problems with a very high sample efficiency. We also introduce a novel method for approximating the prior model from the reward function, which simplifies the expression of complex objectives.
  arXiv  Detail & Related papers  (2021-06-04T10:03:36Z)
• Strictly Batch Imitation Learning by Energy-based Distribution Matching [104.33286163090179]
  Consider learning a policy purely on the basis of demonstrated behavior -- that is, with no access to reinforcement signals, no knowledge of transition dynamics, and no further interaction with the environment. One solution is simply to retrofit existing algorithms for apprenticeship learning to work in the offline setting. But such an approach leans heavily on off-policy evaluation or offline model estimation, and can be indirect and inefficient. We argue that a good solution should be able to explicitly parameterize a policy, implicitly learn from rollout dynamics, and operate in an entirely offline fashion.
  arXiv  Detail & Related papers  (2020-06-25T03:27:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.