Ground-Truthing AI Energy Consumption: Validating CodeCarbon Against External Measurements

• URL: http://arxiv.org/abs/2509.22092v1
• Date: Fri, 26 Sep 2025 09:12:21 GMT
• Title: Ground-Truthing AI Energy Consumption: Validating CodeCarbon Against External Measurements
• Authors: Raphael Fischer,
• Abstract summary: This study systematically evaluates the reliability of static and dynamic energy estimation approaches.<n>The established estimation approaches are shown to consistently make errors of up to 40%.<n>This study establishes transparency and validates widely used tools for sustainable AI development.
• Score: 2.538209532048867
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Although machine learning (ML) and artificial intelligence (AI) present fascinating opportunities for innovation, their rapid development is also significantly impacting our environment. In response to growing resource-awareness in the field, quantification tools such as the ML Emissions Calculator and CodeCarbon were developed to estimate the energy consumption and carbon emissions of running AI models. They are easy to incorporate into AI projects, however also make pragmatic assumptions and neglect important factors, raising the question of estimation accuracy. This study systematically evaluates the reliability of static and dynamic energy estimation approaches through comparisons with ground-truth measurements across hundreds of AI experiments. Based on the proposed validation framework, investigative insights into AI energy demand and estimation inaccuracies are provided. While generally following the patterns of AI energy consumption, the established estimation approaches are shown to consistently make errors of up to 40%. By providing empirical evidence on energy estimation quality and errors, this study establishes transparency and validates widely used tools for sustainable AI development. It moreover formulates guidelines for improving the state-of-the-art and offers code for extending the validation to other domains and tools, thus making important contributions to resource-aware ML and AI sustainability research.

Related papers

• Improving AI Efficiency in Data Centres by Power Dynamic Response [74.12165648170894]
  The steady growth of artificial intelligence (AI) has accelerated in the recent years, facilitated by the development of sophisticated models.<n> Ensuring robust and reliable power infrastructures is fundamental to take advantage of the full potential of AI.<n>However, AI data centres are extremely hungry for power, putting the problem of their power management in the spotlight.
  arXiv  Detail & Related papers  (2025-10-13T08:08:21Z)
• WattsOnAI: Measuring, Analyzing, and Visualizing Energy and Carbon Footprint of AI Workloads [8.545822371190125]
  WattsOnAI is a comprehensive software toolkit for the measurement, analysis, and visualization of energy use, power draw, hardware performance, and carbon emissions across AI workloads.<n>By seamlessly integrating with existing AI frameworks, WattsOnAI offers standardized reports and exports fine-grained time-series data.<n>WattsOnAI encourages the research community to weigh environmental impact alongside raw performance of AI workloads and advances the shift toward more sustainable "Green AI" practices.
  arXiv  Detail & Related papers  (2025-06-25T15:24:45Z)
• General Scales Unlock AI Evaluation with Explanatory and Predictive Power [57.7995945974989]
  benchmarking has guided progress in AI, but it has offered limited explanatory and predictive power for general-purpose AI systems.<n>We introduce general scales for AI evaluation that can explain what common AI benchmarks really measure.<n>Our fully-automated methodology builds on 18 newly-crafted rubrics that place instance demands on general scales that do not saturate.
  arXiv  Detail & Related papers  (2025-03-09T01:13:56Z)
• Generative AI in Health Economics and Outcomes Research: A Taxonomy of Key Definitions and Emerging Applications, an ISPOR Working Group Report [12.204470166456561]
  Generative AI shows significant potential in health economics and outcomes research (HEOR)<n>Generative AI shows significant potential in HEOR, enhancing efficiency, productivity, and offering novel solutions to complex challenges.<n>Foundation models are promising in automating complex tasks, though challenges remain in scientific reliability, bias, interpretability, and workflow integration.
  arXiv  Detail & Related papers  (2024-10-26T15:42:50Z)
• The Responsible Foundation Model Development Cheatsheet: A Review of Tools & Resources [100.23208165760114]
  Foundation model development attracts a rapidly expanding body of contributors, scientists, and applications.<n>To help shape responsible development practices, we introduce the Foundation Model Development Cheatsheet.
  arXiv  Detail & Related papers  (2024-06-24T15:55:49Z)
• Present and Future of AI in Renewable Energy Domain : A Comprehensive Survey [0.0]
  Artificial intelligence (AI) has become a crucial instrument for streamlining processes in various industries. Nine AI-based strategies are identified here to assist Renewable Energy (RE) in contemporary power systems. This study also addressed three main topics: using AI technology for renewable power generation, utilizing AI for renewable energy forecasting, and optimizing energy systems.
  arXiv  Detail & Related papers  (2024-06-22T04:36:09Z)
• Learn to Code Sustainably: An Empirical Study on LLM-based Green Code Generation [7.8273713434806345]
  We evaluate the sustainability of auto-generate codes produced by generative commercial AI language models. We compare the performance and green capacity of human-generated code and code generated by the three AI language models.
  arXiv  Detail & Related papers  (2024-03-05T22:12:01Z)
• On the Opportunities of Green Computing: A Survey [80.21955522431168]
  Artificial Intelligence (AI) has achieved significant advancements in technology and research with the development over several decades. The needs for high computing power brings higher carbon emission and undermines research fairness. To tackle the challenges of computing resources and environmental impact of AI, Green Computing has become a hot research topic.
  arXiv  Detail & Related papers  (2023-11-01T11:16:41Z)
• A Comparative Study of Machine Learning Algorithms for Anomaly Detection in Industrial Environments: Performance and Environmental Impact [62.997667081978825]
  This study seeks to address the demands of high-performance machine learning models with environmental sustainability. Traditional machine learning algorithms, such as Decision Trees and Random Forests, demonstrate robust efficiency and performance. However, superior outcomes were obtained with optimised configurations, albeit with a commensurate increase in resource consumption.
  arXiv  Detail & Related papers  (2023-07-01T15:18:00Z)
• AI Maintenance: A Robustness Perspective [91.28724422822003]
  We introduce highlighted robustness challenges in the AI lifecycle and motivate AI maintenance by making analogies to car maintenance. We propose an AI model inspection framework to detect and mitigate robustness risks. Our proposal for AI maintenance facilitates robustness assessment, status tracking, risk scanning, model hardening, and regulation throughout the AI lifecycle.
  arXiv  Detail & Related papers  (2023-01-08T15:02:38Z)
• The Powerful Use of AI in the Energy Sector: Intelligent Forecasting [7.747343962518897]
  This paper proposes a methodology to develop, deploy, and evaluate AI systems in the energy sector. The goal is to provide a high level of confidence to energy utility users.
  arXiv  Detail & Related papers  (2021-11-03T05:30:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.