Evaluating the Energy Consumption of Machine Learning: Systematic Literature Review and Experiments

• URL: http://arxiv.org/abs/2408.15128v1
• Date: Tue, 27 Aug 2024 15:08:06 GMT
• Title: Evaluating the Energy Consumption of Machine Learning: Systematic Literature Review and Experiments
• Authors: Charlotte Rodriguez, Laura Degioanni, Laetitia Kameni, Richard Vidal, Giovanni Neglia,
• Abstract summary: Monitoring, understanding, and optimizing the energy consumption of Machine Learning (ML) are various reasons why it is necessary to evaluate the energy usage of ML. There exists no universal tool that can answer this question for all use cases, and there may even be disagreement on how to evaluate energy consumption for a specific use case.
• Score: 16.62572282726245
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Monitoring, understanding, and optimizing the energy consumption of Machine Learning (ML) are various reasons why it is necessary to evaluate the energy usage of ML. However, there exists no universal tool that can answer this question for all use cases, and there may even be disagreement on how to evaluate energy consumption for a specific use case. Tools and methods are based on different approaches, each with their own advantages and drawbacks, and they need to be mapped out and explained in order to select the most suitable one for a given situation. We address this challenge through two approaches. First, we conduct a systematic literature review of all tools and methods that permit to evaluate the energy consumption of ML (both at training and at inference), irrespective of whether they were originally designed for machine learning or general software. Second, we develop and use an experimental protocol to compare a selection of these tools and methods. The comparison is both qualitative and quantitative on a range of ML tasks of different nature (vision, language) and computational complexity. The systematic literature review serves as a comprehensive guide for understanding the array of tools and methods used in evaluating energy consumption of ML, for various use cases going from basic energy monitoring to consumption optimization. Two open-source repositories are provided for further exploration. The first one contains tools that can be used to replicate this work or extend the current review. The second repository houses the experimental protocol, allowing users to augment the protocol with new ML computing tasks and additional energy evaluation tools.

Related papers

• Normalizing Energy Consumption for Hardware-Independent Evaluation [9.658615045493734]
  We present a novel methodology for normalizing energy consumption across different hardware platforms. Our approach shows that the number of reference points, the type of regression and the inclusion of computational metrics significantly influences the normalization process.
  arXiv  Detail & Related papers  (2024-09-09T13:38:00Z)
• Towards Completeness-Oriented Tool Retrieval for Large Language Models [60.733557487886635]
  Real-world systems often incorporate a wide array of tools, making it impractical to input all tools into Large Language Models. Existing tool retrieval methods primarily focus on semantic matching between user queries and tool descriptions. We propose a novel modelagnostic COllaborative Learning-based Tool Retrieval approach, COLT, which captures not only the semantic similarities between user queries and tool descriptions but also takes into account the collaborative information of tools.
  arXiv  Detail & Related papers  (2024-05-25T06:41:23Z)
• Towards Coarse-to-Fine Evaluation of Inference Efficiency for Large Language Models [95.96734086126469]
  Large language models (LLMs) can serve as the assistant to help users accomplish their jobs, and also support the development of advanced applications. For the wide application of LLMs, the inference efficiency is an essential concern, which has been widely studied in existing work. We perform a detailed coarse-to-fine analysis of the inference performance of various code libraries.
  arXiv  Detail & Related papers  (2024-04-17T15:57:50Z)
• What Are Tools Anyway? A Survey from the Language Model Perspective [67.18843218893416]
  Language models (LMs) are powerful yet mostly for text generation tasks. We provide a unified definition of tools as external programs used by LMs. We empirically study the efficiency of various tooling methods.
  arXiv  Detail & Related papers  (2024-03-18T17:20:07Z)
• T-Eval: Evaluating the Tool Utilization Capability of Large Language Models Step by Step [69.64348626180623]
  Large language models (LLM) have achieved remarkable performance on various NLP tasks. How to evaluate and analyze the tool-utilization capability of LLMs is still under-explored. We introduce T-Eval to evaluate the tool utilization capability step by step.
  arXiv  Detail & Related papers  (2023-12-21T17:02:06Z)
• Machine Learning for the Multi-Dimensional Bin Packing Problem: Literature Review and Empirical Evaluation [52.560375022430236]
  Bin Packing Problem (BPP) is a well-established optimization (CO) problem. In this article, we first formulate BPP, introducing its variants and practical constraints. Then, a comprehensive survey on machine learning for multi-dimensional BPP is provided.
  arXiv  Detail & Related papers  (2023-12-13T12:39:25Z)
• Utilizing Domain Knowledge: Robust Machine Learning for Building Energy Prediction with Small, Inconsistent Datasets [1.1081836812143175]
  The demand for a huge amount of data for machine learning (ML) applications is currently a bottleneck. We propose a method to combine prior knowledge with data-driven methods to significantly reduce their data dependency. CBML as the knowledge-encoded data-driven method is examined in the context of energy-efficient building engineering.
  arXiv  Detail & Related papers  (2023-01-23T08:56:11Z)
• Topology Optimization via Machine Learning and Deep Learning: A Review [4.447467536572626]
  Topology optimization (TO) is a method of deriving an optimal design that satisfies a given load and boundary conditions within a design domain. This study reviews and analyzes previous research on machine learning-based TO (MLTO)
  arXiv  Detail & Related papers  (2022-10-19T07:47:19Z)
• A Survey on Large-scale Machine Learning [67.6997613600942]
  Machine learning can provide deep insights into data, allowing machines to make high-quality predictions. Most sophisticated machine learning approaches suffer from huge time costs when operating on large-scale data. Large-scale Machine Learning aims to learn patterns from big data with comparable performance efficiently.
  arXiv  Detail & Related papers  (2020-08-10T06:07:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.