PolyThrottle: Energy-efficient Neural Network Inference on Edge Devices

• URL: http://arxiv.org/abs/2310.19991v2
• Date: Tue, 9 Jan 2024 07:13:44 GMT
• Title: PolyThrottle: Energy-efficient Neural Network Inference on Edge Devices
• Authors: Minghao Yan, Hongyi Wang, Shivaram Venkataraman
• Abstract summary: The continuous operation of ML-powered systems leads to significant energy use during inference. This paper investigates how the configuration of on-device hardware-elements such as GPU, memory, and CPU frequency, affects energy consumption for NN inference with regular fine-tuning. We propose PolyThrottle, a solution that optimize configurations across individual hardware components using Constrained Bayesian Optimization in an energy-conserving manner.
• Score: 10.01838504586422
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As neural networks (NN) are deployed across diverse sectors, their energy demand correspondingly grows. While several prior works have focused on reducing energy consumption during training, the continuous operation of ML-powered systems leads to significant energy use during inference. This paper investigates how the configuration of on-device hardware-elements such as GPU, memory, and CPU frequency, often neglected in prior studies, affects energy consumption for NN inference with regular fine-tuning. We propose PolyThrottle, a solution that optimizes configurations across individual hardware components using Constrained Bayesian Optimization in an energy-conserving manner. Our empirical evaluation uncovers novel facets of the energy-performance equilibrium showing that we can save up to 36 percent of energy for popular models. We also validate that PolyThrottle can quickly converge towards near-optimal settings while satisfying application constraints.

Related papers

• Energy-Aware Dynamic Neural Inference [39.04688735618206]
  We introduce an on-device adaptive inference system equipped with an energy-harvester and finite-capacity energy storage. We show that, as the rate of the ambient energy increases, energy- and confidence-aware control schemes show approximately 5% improvement in accuracy. We derive a principled policy with theoretical guarantees for confidence-aware and -agnostic controllers.
  arXiv  Detail & Related papers  (2024-11-04T16:51:22Z)
• Revisiting DNN Training for Intermittently Powered Energy Harvesting Micro Computers [0.6721767679705013]
  This study introduces and evaluates a novel training methodology tailored for Deep Neural Networks in energy-constrained environments. We propose a dynamic dropout technique that adapts to both the architecture of the device and the variability in energy availability. Preliminary results demonstrate that this strategy provides 6 to 22 percent accuracy improvements compared to the state of the art with less than 5 percent additional compute.
  arXiv  Detail & Related papers  (2024-08-25T01:13:00Z)
• Federated Learning With Energy Harvesting Devices: An MDP Framework [5.852486435612777]
  Federated learning (FL) requires edge devices to perform local training and exchange information with a parameter server. A critical challenge in practical FL systems is the rapid energy depletion of battery-limited edge devices. We apply energy harvesting technique in FL systems to extract ambient energy for continuously powering edge devices.
  arXiv  Detail & Related papers  (2024-05-17T03:41:40Z)
• Sustainable Edge Intelligence Through Energy-Aware Early Exiting [0.726437825413781]
  We propose energy-adaptive dynamic early exiting to enable efficient and accurate inference in an EH edge intelligence system. Our approach derives an energy-aware EE policy that determines the optimal amount of computational processing on a per-sample basis. Results show that accuracy and service rate are improved up to 25% and 35%, respectively, in comparison with an energy-agnostic policy.
  arXiv  Detail & Related papers  (2023-05-23T14:17:44Z)
• Distributed Energy Management and Demand Response in Smart Grids: A Multi-Agent Deep Reinforcement Learning Framework [53.97223237572147]
  This paper presents a multi-agent Deep Reinforcement Learning (DRL) framework for autonomous control and integration of renewable energy resources into smart power grid systems. In particular, the proposed framework jointly considers demand response (DR) and distributed energy management (DEM) for residential end-users.
  arXiv  Detail & Related papers  (2022-11-29T01:18:58Z)
• Enforcing Policy Feasibility Constraints through Differentiable Projection for Energy Optimization [57.88118988775461]
  We propose PROjected Feasibility (PROF) to enforce convex operational constraints within neural policies. We demonstrate PROF on two applications: energy-efficient building operation and inverter control.
  arXiv  Detail & Related papers  (2021-05-19T01:58:10Z)
• Threshold-Based Data Exclusion Approach for Energy-Efficient Federated Edge Learning [4.25234252803357]
  Federated edge learning (FEEL) is a promising distributed learning technique for next-generation wireless networks. FEEL might significantly shorten energy-constrained participating devices' lifetime due to the power consumed during the model training round. This paper proposes a novel approach that endeavors to minimize computation and communication energy consumption during FEEL rounds.
  arXiv  Detail & Related papers  (2021-03-30T13:34:40Z)
• To Talk or to Work: Flexible Communication Compression for Energy Efficient Federated Learning over Heterogeneous Mobile Edge Devices [78.38046945665538]
  federated learning (FL) over massive mobile edge devices opens new horizons for numerous intelligent mobile applications. FL imposes huge communication and computation burdens on participating devices due to periodical global synchronization and continuous local training. We develop a convergence-guaranteed FL algorithm enabling flexible communication compression.
  arXiv  Detail & Related papers  (2020-12-22T02:54:18Z)
• Energy-Based Processes for Exchangeable Data [109.04978766553612]
  We introduce Energy-Based Processes (EBPs) to extend energy based models to exchangeable data. A key advantage of EBPs is the ability to express more flexible distributions over sets without restricting their cardinality. We develop an efficient training procedure for EBPs that demonstrates state-of-the-art performance on a variety of tasks.
  arXiv  Detail & Related papers  (2020-03-17T04:26:02Z)
• Risk-Aware Energy Scheduling for Edge Computing with Microgrid: A Multi-Agent Deep Reinforcement Learning Approach [82.6692222294594]
  We study a risk-aware energy scheduling problem for a microgrid-powered MEC network. We derive the solution by applying a multi-agent deep reinforcement learning (MADRL)-based advantage actor-critic (A3C) algorithm with shared neural networks.
  arXiv  Detail & Related papers  (2020-02-21T02:14:38Z)
• Multi-Agent Meta-Reinforcement Learning for Self-Powered and Sustainable Edge Computing Systems [87.4519172058185]
  An effective energy dispatch mechanism for self-powered wireless networks with edge computing capabilities is studied. A novel multi-agent meta-reinforcement learning (MAMRL) framework is proposed to solve the formulated problem. Experimental results show that the proposed MAMRL model can reduce up to 11% non-renewable energy usage and by 22.4% the energy cost.
  arXiv  Detail & Related papers  (2020-02-20T04:58:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.