Unveiling Energy Efficiency in Deep Learning: Measurement, Prediction, and Scoring across Edge Devices

• URL: http://arxiv.org/abs/2310.18329v2
• Date: Mon, 10 Jun 2024 15:09:24 GMT
• Title: Unveiling Energy Efficiency in Deep Learning: Measurement, Prediction, and Scoring across Edge Devices
• Authors: Xiaolong Tu, Anik Mallik, Dawei Chen, Kyungtae Han, Onur Altintas, Haoxin Wang, Jiang Xie,
• Abstract summary: We conduct a threefold study, including energy measurement, prediction, and efficiency scoring. Firstly, we present a detailed, first-of-its-kind measurement study that uncovers the energy consumption characteristics of on-device deep learning. Secondly, we design and implement the first kernel-level energy predictors for edge devices based on our kernel-level energy dataset.
• Score: 8.140572894424208
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Today, deep learning optimization is primarily driven by research focused on achieving high inference accuracy and reducing latency. However, the energy efficiency aspect is often overlooked, possibly due to a lack of sustainability mindset in the field and the absence of a holistic energy dataset. In this paper, we conduct a threefold study, including energy measurement, prediction, and efficiency scoring, with an objective to foster transparency in power and energy consumption within deep learning across various edge devices. Firstly, we present a detailed, first-of-its-kind measurement study that uncovers the energy consumption characteristics of on-device deep learning. This study results in the creation of three extensive energy datasets for edge devices, covering a wide range of kernels, state-of-the-art DNN models, and popular AI applications. Secondly, we design and implement the first kernel-level energy predictors for edge devices based on our kernel-level energy dataset. Evaluation results demonstrate the ability of our predictors to provide consistent and accurate energy estimations on unseen DNN models. Lastly, we introduce two scoring metrics, PCS and IECS, developed to convert complex power and energy consumption data of an edge device into an easily understandable manner for edge device end-users. We hope our work can help shift the mindset of both end-users and the research community towards sustainability in edge computing, a principle that drives our research. Find data, code, and more up-to-date information at https://amai-gsu.github.io/DeepEn2023.

Related papers

• Energy-Aware Dynamic Neural Inference [39.04688735618206]
  We introduce an on-device adaptive inference system equipped with an energy-harvester and finite-capacity energy storage. We show that, as the rate of the ambient energy increases, energy- and confidence-aware control schemes show approximately 5% improvement in accuracy. We derive a principled policy with theoretical guarantees for confidence-aware and -agnostic controllers.
  arXiv  Detail & Related papers  (2024-11-04T16:51:22Z)
• Measuring the Energy Consumption and Efficiency of Deep Neural Networks: An Empirical Analysis and Design Recommendations [0.49478969093606673]
  BUTTER-E dataset is an augmentation to the BUTTER Empirical Deep Learning dataset. This dataset reveals the complex relationship between dataset size, network structure, and energy use. We propose a straightforward and effective energy model that accounts for network size, computing, and memory hierarchy.
  arXiv  Detail & Related papers  (2024-03-13T00:27:19Z)
• Green Edge AI: A Contemporary Survey [46.11332733210337]
  The transformative power of AI is derived from the utilization of deep neural networks (DNNs) Deep learning (DL) is increasingly being transitioned to wireless edge networks in proximity to end-user devices (EUDs) Despite its potential, edge AI faces substantial challenges, mostly due to the dichotomy between the resource limitations of wireless edge networks and the resource-intensive nature of DL.
  arXiv  Detail & Related papers  (2023-12-01T04:04:37Z)
• Computation-efficient Deep Learning for Computer Vision: A Survey [121.84121397440337]
  Deep learning models have reached or even exceeded human-level performance in a range of visual perception tasks. Deep learning models usually demand significant computational resources, leading to impractical power consumption, latency, or carbon emissions in real-world scenarios. New research focus is computationally efficient deep learning, which strives to achieve satisfactory performance while minimizing the computational cost during inference.
  arXiv  Detail & Related papers  (2023-08-27T03:55:28Z)
• Sustainable Edge Intelligence Through Energy-Aware Early Exiting [0.726437825413781]
  We propose energy-adaptive dynamic early exiting to enable efficient and accurate inference in an EH edge intelligence system. Our approach derives an energy-aware EE policy that determines the optimal amount of computational processing on a per-sample basis. Results show that accuracy and service rate are improved up to 25% and 35%, respectively, in comparison with an energy-agnostic policy.
  arXiv  Detail & Related papers  (2023-05-23T14:17:44Z)
• Uncovering Energy-Efficient Practices in Deep Learning Training: Preliminary Steps Towards Green AI [8.025202812165412]
  We consider energy consumption as a metric of equal importance to accuracy and to reduce any irrelevant tasks or energy usage. We examine the training stage of the deep learning pipeline from a sustainability perspective. We highlight innovative and promising energy-efficient practices for training deep learning models.
  arXiv  Detail & Related papers  (2023-03-24T12:48:21Z)
• Energy Consumption of Neural Networks on NVIDIA Edge Boards: an Empirical Model [6.809944967863927]
  Recently, there has been a trend of shifting the execution of deep learning inference tasks toward the edge of the network, closer to the user, to reduce latency and preserve data privacy. In this work, we aim at profiling the energetic consumption of inference tasks for some modern edge nodes. We have then distilled a simple, practical model that can provide an estimate of the energy consumption of a certain inference task on the considered boards.
  arXiv  Detail & Related papers  (2022-10-04T14:12:59Z)
• Learning, Computing, and Trustworthiness in Intelligent IoT Environments: Performance-Energy Tradeoffs [62.91362897985057]
  An Intelligent IoT Environment (iIoTe) is comprised of heterogeneous devices that can collaboratively execute semi-autonomous IoT applications. This paper provides a state-of-the-art overview of these technologies and illustrates their functionality and performance, with special attention to the tradeoff among resources, latency, privacy and energy consumption.
  arXiv  Detail & Related papers  (2021-10-04T19:41:42Z)
• Compute and Energy Consumption Trends in Deep Learning Inference [67.32875669386488]
  We study relevant models in the areas of computer vision and natural language processing. For a sustained increase in performance we see a much softer growth in energy consumption than previously anticipated.
  arXiv  Detail & Related papers  (2021-09-12T09:40:18Z)
• Energy Drain of the Object Detection Processing Pipeline for Mobile Devices: Analysis and Implications [77.00418462388525]
  This paper presents the first detailed experimental study of a mobile augmented reality (AR) client's energy consumption and the detection latency of executing Convolutional Neural Networks (CNN) based object detection. Our detailed measurements refine the energy analysis of mobile AR clients and reveal several interesting perspectives regarding the energy consumption of executing CNN-based object detection.
  arXiv  Detail & Related papers  (2020-11-26T00:32:07Z)
• Towards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learning [68.37641996188133]
  We introduce a framework for tracking realtime energy consumption and carbon emissions. We create a leaderboard for energy efficient reinforcement learning algorithms. We propose strategies for mitigation of carbon emissions and reduction of energy consumption.
  arXiv  Detail & Related papers  (2020-01-31T05:12:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.