Towards Accurate and Reliable Energy Measurement of NLP Models

• URL: http://arxiv.org/abs/2010.05248v1
• Date: Sun, 11 Oct 2020 13:44:52 GMT
• Title: Towards Accurate and Reliable Energy Measurement of NLP Models
• Authors: Qingqing Cao, Aruna Balasubramanian, Niranjan Balasubramanian
• Abstract summary: We show that existing software-based energy measurements are not accurate because they do not take into account hardware differences and how resource utilization affects energy consumption. We quantify the error of existing software-based energy measurements by using a hardware power meter that provides highly accurate energy measurements. Our key takeaway is the need for a more accurate energy estimation model that takes into account hardware variabilities and the non-linear relationship between resource utilization and energy consumption.
• Score: 20.289537200662306
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Accurate and reliable measurement of energy consumption is critical for making well-informed design choices when choosing and training large scale NLP models. In this work, we show that existing software-based energy measurements are not accurate because they do not take into account hardware differences and how resource utilization affects energy consumption. We conduct energy measurement experiments with four different models for a question answering task. We quantify the error of existing software-based energy measurements by using a hardware power meter that provides highly accurate energy measurements. Our key takeaway is the need for a more accurate energy estimation model that takes into account hardware variabilities and the non-linear relationship between resource utilization and energy consumption. We release the code and data at https://github.com/csarron/sustainlp2020-energy.

Related papers

• Enhancing Energy-Awareness in Deep Learning through Fine-Grained Energy Measurement [11.37120215795946]
  This paper introduces FECoM (Fine-grained Energy Consumption Meter), a framework for fine-grained Deep Learning energy consumption measurement. FECoM addresses the challenges of measuring energy consumption at fine-grained level by using static instrumentation and considering various factors, including computational load stability and temperature.
  arXiv  Detail & Related papers  (2023-08-23T17:32:06Z)
• ecoBLE: A Low-Computation Energy Consumption Prediction Framework for Bluetooth Low Energy [9.516475567386768]
  Bluetooth Low Energy (BLE) is a de-facto technology for Internet of Things (IoT) applications, promising very low energy consumption. This paper introduces a Long Short-Term Memory Projection (LSTMP)-based BLE energy consumption prediction framework. Our results show that the proposed framework predicts the energy consumption of different BLE nodes with a Mean Absolute Percentage Error (MAPE) of up to 12%.
  arXiv  Detail & Related papers  (2023-08-02T13:04:23Z)
• Counting Carbon: A Survey of Factors Influencing the Emissions of Machine Learning [77.62876532784759]
  Machine learning (ML) requires using energy to carry out computations during the model training process. The generation of this energy comes with an environmental cost in terms of greenhouse gas emissions, depending on quantity used and the energy source. We present a survey of the carbon emissions of 95 ML models across time and different tasks in natural language processing and computer vision.
  arXiv  Detail & Related papers  (2023-02-16T18:35:00Z)
• Guaranteed efficient energy estimation of quantum many-body Hamiltonians using ShadowGrouping [55.47824411563162]
  Estimation of the energy of quantum many-body systems is a paradigmatic task in various research fields. We aim to find the optimal strategy with single-qubit measurements that yields the highest provable accuracy given a total measurement budget. We develop a practical, efficient estimation strategy, which we call ShadowGrouping.
  arXiv  Detail & Related papers  (2023-01-09T14:41:07Z)
• Precise Energy Consumption Measurements of Heterogeneous Artificial Intelligence Workloads [0.534434568021034]
  We present measurements of the energy consumption of two typical applications of deep learning models on different types of compute nodes. One advantage of our approach is that the information on energy consumption is available to all users of the supercomputer.
  arXiv  Detail & Related papers  (2022-12-03T21:40:55Z)
• IrEne: Interpretable Energy Prediction for Transformers [15.677294441315535]
  Existing software-based energy measurements of NLP models are not accurate because they do not consider the complex interactions between energy consumption and model execution. We present IrEne, an interpretable and energy prediction system that accurately predicts the inference energy consumption of a wide range of Transformer-based NLP models.
  arXiv  Detail & Related papers  (2021-06-02T14:43:51Z)
• EnergyVis: Interactively Tracking and Exploring Energy Consumption for ML Models [8.939420322774243]
  EnergyVis is an interactive energy consumption tracker for machine learning (ML) models. It enables researchers to interactively track, visualize and compare model energy consumption across key energy consumption and carbon footprint metrics. EnergyVis aims to raise awareness concerning computational sustainability by interactively highlighting excessive energy usage during model training.
  arXiv  Detail & Related papers  (2021-03-30T15:33:43Z)
• Power Modeling for Effective Datacenter Planning and Compute Management [53.41102502425513]
  We discuss two classes of statistical power models designed and validated to be accurate, simple, interpretable and applicable to all hardware configurations and workloads. We demonstrate that the proposed statistical modeling techniques, while simple and scalable, predict power with less than 5% Mean Absolute Percent Error (MAPE) for more than 95% diverse Power Distribution Units (more than 2000) using only 4 features.
  arXiv  Detail & Related papers  (2021-03-22T21:22:51Z)
• Energy Drain of the Object Detection Processing Pipeline for Mobile Devices: Analysis and Implications [77.00418462388525]
  This paper presents the first detailed experimental study of a mobile augmented reality (AR) client's energy consumption and the detection latency of executing Convolutional Neural Networks (CNN) based object detection. Our detailed measurements refine the energy analysis of mobile AR clients and reveal several interesting perspectives regarding the energy consumption of executing CNN-based object detection.
  arXiv  Detail & Related papers  (2020-11-26T00:32:07Z)
• Risk-Aware Energy Scheduling for Edge Computing with Microgrid: A Multi-Agent Deep Reinforcement Learning Approach [82.6692222294594]
  We study a risk-aware energy scheduling problem for a microgrid-powered MEC network. We derive the solution by applying a multi-agent deep reinforcement learning (MADRL)-based advantage actor-critic (A3C) algorithm with shared neural networks.
  arXiv  Detail & Related papers  (2020-02-21T02:14:38Z)
• Multi-Agent Meta-Reinforcement Learning for Self-Powered and Sustainable Edge Computing Systems [87.4519172058185]
  An effective energy dispatch mechanism for self-powered wireless networks with edge computing capabilities is studied. A novel multi-agent meta-reinforcement learning (MAMRL) framework is proposed to solve the formulated problem. Experimental results show that the proposed MAMRL model can reduce up to 11% non-renewable energy usage and by 22.4% the energy cost.
  arXiv  Detail & Related papers  (2020-02-20T04:58:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.