Compute and Energy Consumption Trends in Deep Learning Inference
- URL: http://arxiv.org/abs/2109.05472v2
- Date: Wed, 29 Mar 2023 11:34:30 GMT
- Title: Compute and Energy Consumption Trends in Deep Learning Inference
- Authors: Radosvet Desislavov, Fernando Mart\'inez-Plumed, Jos\'e
Hern\'andez-Orallo
- Abstract summary: We study relevant models in the areas of computer vision and natural language processing.
For a sustained increase in performance we see a much softer growth in energy consumption than previously anticipated.
- Score: 67.32875669386488
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: The progress of some AI paradigms such as deep learning is said to be linked
to an exponential growth in the number of parameters. There are many studies
corroborating these trends, but does this translate into an exponential
increase in energy consumption? In order to answer this question we focus on
inference costs rather than training costs, as the former account for most of
the computing effort, solely because of the multiplicative factors. Also, apart
from algorithmic innovations, we account for more specific and powerful
hardware (leading to higher FLOPS) that is usually accompanied with important
energy efficiency optimisations. We also move the focus from the first
implementation of a breakthrough paper towards the consolidated version of the
techniques one or two year later. Under this distinctive and comprehensive
perspective, we study relevant models in the areas of computer vision and
natural language processing: for a sustained increase in performance we see a
much softer growth in energy consumption than previously anticipated. The only
caveat is, yet again, the multiplicative factor, as future AI increases
penetration and becomes more pervasive.
Related papers
- Hype, Sustainability, and the Price of the Bigger-is-Better Paradigm in AI [67.58673784790375]
We argue that the 'bigger is better' AI paradigm is not only fragile scientifically, but comes with undesirable consequences.
First, it is not sustainable, as its compute demands increase faster than model performance, leading to unreasonable economic requirements and a disproportionate environmental footprint.
Second, it implies focusing on certain problems at the expense of others, leaving aside important applications, e.g. health, education, or the climate.
arXiv Detail & Related papers (2024-09-21T14:43:54Z) - Learning Iterative Reasoning through Energy Diffusion [90.24765095498392]
We introduce iterative reasoning through energy diffusion (IRED), a novel framework for learning to reason for a variety of tasks.
IRED learns energy functions to represent the constraints between input conditions and desired outputs.
We show IRED outperforms existing methods in continuous-space reasoning, discrete-space reasoning, and planning tasks.
arXiv Detail & Related papers (2024-06-17T03:36:47Z) - Landscape-Aware Growing: The Power of a Little LAG [49.897766925371485]
We study the question of how to select the best growing strategy from a given pool of growing strategies.
We present an alternative perspective based on early training dynamics, which we call "landscape-aware growing (LAG)"
arXiv Detail & Related papers (2024-06-04T16:38:57Z) - On the Opportunities of Green Computing: A Survey [80.21955522431168]
Artificial Intelligence (AI) has achieved significant advancements in technology and research with the development over several decades.
The needs for high computing power brings higher carbon emission and undermines research fairness.
To tackle the challenges of computing resources and environmental impact of AI, Green Computing has become a hot research topic.
arXiv Detail & Related papers (2023-11-01T11:16:41Z) - Unveiling Energy Efficiency in Deep Learning: Measurement, Prediction, and Scoring across Edge Devices [8.140572894424208]
We conduct a threefold study, including energy measurement, prediction, and efficiency scoring.
Firstly, we present a detailed, first-of-its-kind measurement study that uncovers the energy consumption characteristics of on-device deep learning.
Secondly, we design and implement the first kernel-level energy predictors for edge devices based on our kernel-level energy dataset.
arXiv Detail & Related papers (2023-10-19T23:55:00Z) - Minimizing Energy Consumption of Deep Learning Models by Energy-Aware
Training [26.438415753870917]
We propose EAT, a gradient-based algorithm that aims to reduce energy consumption during model training.
We demonstrate that our energy-aware training algorithm EAT is able to train networks with a better trade-off between classification performance and energy efficiency.
arXiv Detail & Related papers (2023-07-01T15:44:01Z) - Uncovering Energy-Efficient Practices in Deep Learning Training:
Preliminary Steps Towards Green AI [8.025202812165412]
We consider energy consumption as a metric of equal importance to accuracy and to reduce any irrelevant tasks or energy usage.
We examine the training stage of the deep learning pipeline from a sustainability perspective.
We highlight innovative and promising energy-efficient practices for training deep learning models.
arXiv Detail & Related papers (2023-03-24T12:48:21Z) - Precise Energy Consumption Measurements of Heterogeneous Artificial
Intelligence Workloads [0.534434568021034]
We present measurements of the energy consumption of two typical applications of deep learning models on different types of compute nodes.
One advantage of our approach is that the information on energy consumption is available to all users of the supercomputer.
arXiv Detail & Related papers (2022-12-03T21:40:55Z) - Attention Mechanism with Energy-Friendly Operations [61.58748425876866]
We rethink attention mechanism from the energy consumption aspects.
We build a novel attention model by replacing multiplications with either selective operations or additions.
Empirical results on three machine translation tasks demonstrate that the proposed model achieves competitable accuracy.
arXiv Detail & Related papers (2022-04-28T08:50:09Z) - Boosting algorithms in energy research: A systematic review [0.0]
Boosting algorithms are characterized by both high flexibility and high interpretability.
We show that boosting has been underexploited so far, while great advances in the energy field are possible.
arXiv Detail & Related papers (2020-04-01T18:03:26Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.