Measuring the Energy Consumption and Efficiency of Deep Neural Networks: An Empirical Analysis and Design Recommendations

• URL: http://arxiv.org/abs/2403.08151v1
• Date: Wed, 13 Mar 2024 00:27:19 GMT
• Title: Measuring the Energy Consumption and Efficiency of Deep Neural Networks: An Empirical Analysis and Design Recommendations
• Authors: Charles Edison Tripp, Jordan Perr-Sauer, Jamil Gafur, Amabarish Nag, Avi Purkayastha, Sagi Zisman, Erik A. Bensen
• Abstract summary: BUTTER-E dataset is an augmentation to the BUTTER Empirical Deep Learning dataset. This dataset reveals the complex relationship between dataset size, network structure, and energy use. We propose a straightforward and effective energy model that accounts for network size, computing, and memory hierarchy.
• Score: 0.49478969093606673
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Addressing the so-called ``Red-AI'' trend of rising energy consumption by large-scale neural networks, this study investigates the actual energy consumption, as measured by node-level watt-meters, of training various fully connected neural network architectures. We introduce the BUTTER-E dataset, an augmentation to the BUTTER Empirical Deep Learning dataset, containing energy consumption and performance data from 63,527 individual experimental runs spanning 30,582 distinct configurations: 13 datasets, 20 sizes (number of trainable parameters), 8 network ``shapes'', and 14 depths on both CPU and GPU hardware collected using node-level watt-meters. This dataset reveals the complex relationship between dataset size, network structure, and energy use, and highlights the impact of cache effects. We propose a straightforward and effective energy model that accounts for network size, computing, and memory hierarchy. Our analysis also uncovers a surprising, hardware-mediated non-linear relationship between energy efficiency and network design, challenging the assumption that reducing the number of parameters or FLOPs is the best way to achieve greater energy efficiency. Highlighting the need for cache-considerate algorithm development, we suggest a combined approach to energy efficient network, algorithm, and hardware design. This work contributes to the fields of sustainable computing and Green AI, offering practical guidance for creating more energy-efficient neural networks and promoting sustainable AI.

Related papers

• Task-Oriented Real-time Visual Inference for IoVT Systems: A Co-design Framework of Neural Networks and Edge Deployment [61.20689382879937]
  Task-oriented edge computing addresses this by shifting data analysis to the edge. Existing methods struggle to balance high model performance with low resource consumption. We propose a novel co-design framework to optimize neural network architecture.
  arXiv  Detail & Related papers  (2024-10-29T19:02:54Z)
• From Computation to Consumption: Exploring the Compute-Energy Link for Training and Testing Neural Networks for SED Systems [9.658615045493734]
  We study several neural network architectures that are key components of sound event detection systems. We measure the energy consumption for training and testing small to large architectures. We establish complex relationships between the energy consumption, the number of floating-point operations, the number of parameters, and the GPU/memory utilization.
  arXiv  Detail & Related papers  (2024-09-08T12:51:34Z)
• Revisiting DNN Training for Intermittently Powered Energy Harvesting Micro Computers [0.6721767679705013]
  This study introduces and evaluates a novel training methodology tailored for Deep Neural Networks in energy-constrained environments. We propose a dynamic dropout technique that adapts to both the architecture of the device and the variability in energy availability. Preliminary results demonstrate that this strategy provides 6 to 22 percent accuracy improvements compared to the state of the art with less than 5 percent additional compute.
  arXiv  Detail & Related papers  (2024-08-25T01:13:00Z)
• Data-driven Energy Efficiency Modelling in Large-scale Networks: An Expert Knowledge and ML-based Approach [8.326834499339107]
  This paper introduces the simulated reality of communication networks (SRCON) framework. It harnesses live network data and employs a blend of machine learning (ML)- and expert-based models. Results show significant gains over a state-of-the art method used by a operator for network energy efficiency modeling.
  arXiv  Detail & Related papers  (2023-12-31T10:03:08Z)
• Unveiling Energy Efficiency in Deep Learning: Measurement, Prediction, and Scoring across Edge Devices [8.140572894424208]
  We conduct a threefold study, including energy measurement, prediction, and efficiency scoring. Firstly, we present a detailed, first-of-its-kind measurement study that uncovers the energy consumption characteristics of on-device deep learning. Secondly, we design and implement the first kernel-level energy predictors for edge devices based on our kernel-level energy dataset.
  arXiv  Detail & Related papers  (2023-10-19T23:55:00Z)
• Computation-efficient Deep Learning for Computer Vision: A Survey [121.84121397440337]
  Deep learning models have reached or even exceeded human-level performance in a range of visual perception tasks. Deep learning models usually demand significant computational resources, leading to impractical power consumption, latency, or carbon emissions in real-world scenarios. New research focus is computationally efficient deep learning, which strives to achieve satisfactory performance while minimizing the computational cost during inference.
  arXiv  Detail & Related papers  (2023-08-27T03:55:28Z)
• Energy Consumption of Neural Networks on NVIDIA Edge Boards: an Empirical Model [6.809944967863927]
  Recently, there has been a trend of shifting the execution of deep learning inference tasks toward the edge of the network, closer to the user, to reduce latency and preserve data privacy. In this work, we aim at profiling the energetic consumption of inference tasks for some modern edge nodes. We have then distilled a simple, practical model that can provide an estimate of the energy consumption of a certain inference task on the considered boards.
  arXiv  Detail & Related papers  (2022-10-04T14:12:59Z)
• Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks [78.47459801017959]
  Sparsity can reduce the memory footprint of regular networks to fit mobile devices. We describe approaches to remove and add elements of neural networks, different training strategies to achieve model sparsity, and mechanisms to exploit sparsity in practice.
  arXiv  Detail & Related papers  (2021-01-31T22:48:50Z)
• Energy Drain of the Object Detection Processing Pipeline for Mobile Devices: Analysis and Implications [77.00418462388525]
  This paper presents the first detailed experimental study of a mobile augmented reality (AR) client's energy consumption and the detection latency of executing Convolutional Neural Networks (CNN) based object detection. Our detailed measurements refine the energy analysis of mobile AR clients and reveal several interesting perspectives regarding the energy consumption of executing CNN-based object detection.
  arXiv  Detail & Related papers  (2020-11-26T00:32:07Z)
• Dynamic Graph: Learning Instance-aware Connectivity for Neural Networks [78.65792427542672]
  Dynamic Graph Network (DG-Net) is a complete directed acyclic graph, where the nodes represent convolutional blocks and the edges represent connection paths. Instead of using the same path of the network, DG-Net aggregates features dynamically in each node, which allows the network to have more representation ability.
  arXiv  Detail & Related papers  (2020-10-02T16:50:26Z)
• DepthNet Nano: A Highly Compact Self-Normalizing Neural Network for Monocular Depth Estimation [76.90627702089357]
  DepthNet Nano is a compact deep neural network for monocular depth estimation designed using a human machine collaborative design strategy. The proposed DepthNet Nano possesses a highly efficient network architecture, while still achieving comparable performance with state-of-the-art networks.
  arXiv  Detail & Related papers  (2020-04-17T00:41:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.