PolyThrottle: Energy-efficient Neural Network Inference on Edge Devices
- URL: http://arxiv.org/abs/2310.19991v2
- Date: Tue, 9 Jan 2024 07:13:44 GMT
- Title: PolyThrottle: Energy-efficient Neural Network Inference on Edge Devices
- Authors: Minghao Yan, Hongyi Wang, Shivaram Venkataraman
- Abstract summary: The continuous operation of ML-powered systems leads to significant energy use during inference.
This paper investigates how the configuration of on-device hardware-elements such as GPU, memory, and CPU frequency, affects energy consumption for NN inference with regular fine-tuning.
We propose PolyThrottle, a solution that optimize configurations across individual hardware components using Constrained Bayesian Optimization in an energy-conserving manner.
- Score: 10.01838504586422
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: As neural networks (NN) are deployed across diverse sectors, their energy
demand correspondingly grows. While several prior works have focused on
reducing energy consumption during training, the continuous operation of
ML-powered systems leads to significant energy use during inference. This paper
investigates how the configuration of on-device hardware-elements such as GPU,
memory, and CPU frequency, often neglected in prior studies, affects energy
consumption for NN inference with regular fine-tuning. We propose PolyThrottle,
a solution that optimizes configurations across individual hardware components
using Constrained Bayesian Optimization in an energy-conserving manner. Our
empirical evaluation uncovers novel facets of the energy-performance
equilibrium showing that we can save up to 36 percent of energy for popular
models. We also validate that PolyThrottle can quickly converge towards
near-optimal settings while satisfying application constraints.
Related papers
- Federated Learning With Energy Harvesting Devices: An MDP Framework [5.852486435612777]
Federated learning (FL) requires edge devices to perform local training and exchange information with a parameter server.
A critical challenge in practical FL systems is the rapid energy depletion of battery-limited edge devices.
We apply energy harvesting technique in FL systems to extract ambient energy for continuously powering edge devices.
arXiv Detail & Related papers (2024-05-17T03:41:40Z) - Unveiling Energy Efficiency in Deep Learning: Measurement, Prediction, and Scoring across Edge Devices [8.140572894424208]
We conduct a threefold study, including energy measurement, prediction, and efficiency scoring.
Firstly, we present a detailed, first-of-its-kind measurement study that uncovers the energy consumption characteristics of on-device deep learning.
Secondly, we design and implement the first kernel-level energy predictors for edge devices based on our kernel-level energy dataset.
arXiv Detail & Related papers (2023-10-19T23:55:00Z) - Sustainable Edge Intelligence Through Energy-Aware Early Exiting [0.726437825413781]
We propose energy-adaptive dynamic early exiting to enable efficient and accurate inference in an EH edge intelligence system.
Our approach derives an energy-aware EE policy that determines the optimal amount of computational processing on a per-sample basis.
Results show that accuracy and service rate are improved up to 25% and 35%, respectively, in comparison with an energy-agnostic policy.
arXiv Detail & Related papers (2023-05-23T14:17:44Z) - Distributed Energy Management and Demand Response in Smart Grids: A
Multi-Agent Deep Reinforcement Learning Framework [53.97223237572147]
This paper presents a multi-agent Deep Reinforcement Learning (DRL) framework for autonomous control and integration of renewable energy resources into smart power grid systems.
In particular, the proposed framework jointly considers demand response (DR) and distributed energy management (DEM) for residential end-users.
arXiv Detail & Related papers (2022-11-29T01:18:58Z) - Enforcing Policy Feasibility Constraints through Differentiable
Projection for Energy Optimization [57.88118988775461]
We propose PROjected Feasibility (PROF) to enforce convex operational constraints within neural policies.
We demonstrate PROF on two applications: energy-efficient building operation and inverter control.
arXiv Detail & Related papers (2021-05-19T01:58:10Z) - Threshold-Based Data Exclusion Approach for Energy-Efficient Federated
Edge Learning [4.25234252803357]
Federated edge learning (FEEL) is a promising distributed learning technique for next-generation wireless networks.
FEEL might significantly shorten energy-constrained participating devices' lifetime due to the power consumed during the model training round.
This paper proposes a novel approach that endeavors to minimize computation and communication energy consumption during FEEL rounds.
arXiv Detail & Related papers (2021-03-30T13:34:40Z) - To Talk or to Work: Flexible Communication Compression for Energy
Efficient Federated Learning over Heterogeneous Mobile Edge Devices [78.38046945665538]
federated learning (FL) over massive mobile edge devices opens new horizons for numerous intelligent mobile applications.
FL imposes huge communication and computation burdens on participating devices due to periodical global synchronization and continuous local training.
We develop a convergence-guaranteed FL algorithm enabling flexible communication compression.
arXiv Detail & Related papers (2020-12-22T02:54:18Z) - Intermittent Inference with Nonuniformly Compressed Multi-Exit Neural
Network for Energy Harvesting Powered Devices [17.165614326127287]
This work aims to enable persistent, event-driven sensing and decision capabilities for energy-harvesting (EH)-powered devices.
We developed a power trace-aware and exit-guided network compression algorithm to compress and deploy multi-exit neural networks to EH-powered microcontrollers.
Experiments show superior accuracy and latency compared with state-of-the-art techniques.
arXiv Detail & Related papers (2020-04-23T16:19:22Z) - Energy-Based Processes for Exchangeable Data [109.04978766553612]
We introduce Energy-Based Processes (EBPs) to extend energy based models to exchangeable data.
A key advantage of EBPs is the ability to express more flexible distributions over sets without restricting their cardinality.
We develop an efficient training procedure for EBPs that demonstrates state-of-the-art performance on a variety of tasks.
arXiv Detail & Related papers (2020-03-17T04:26:02Z) - Risk-Aware Energy Scheduling for Edge Computing with Microgrid: A
Multi-Agent Deep Reinforcement Learning Approach [82.6692222294594]
We study a risk-aware energy scheduling problem for a microgrid-powered MEC network.
We derive the solution by applying a multi-agent deep reinforcement learning (MADRL)-based advantage actor-critic (A3C) algorithm with shared neural networks.
arXiv Detail & Related papers (2020-02-21T02:14:38Z) - Multi-Agent Meta-Reinforcement Learning for Self-Powered and Sustainable
Edge Computing Systems [87.4519172058185]
An effective energy dispatch mechanism for self-powered wireless networks with edge computing capabilities is studied.
A novel multi-agent meta-reinforcement learning (MAMRL) framework is proposed to solve the formulated problem.
Experimental results show that the proposed MAMRL model can reduce up to 11% non-renewable energy usage and by 22.4% the energy cost.
arXiv Detail & Related papers (2020-02-20T04:58:07Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.