Prompt engineering and its implications on the energy consumption of Large Language Models

• URL: http://arxiv.org/abs/2501.05899v1
• Date: Fri, 10 Jan 2025 11:49:31 GMT
• Title: Prompt engineering and its implications on the energy consumption of Large Language Models
• Authors: Riccardo Rubei, Aicha Moussaid, Claudio di Sipio, Davide di Ruscio,
• Abstract summary: Large language models (LLMs) in software engineering pose severe challenges regarding computational resources, data centers, and carbon emissions.<n>In this paper, we investigate how prompt engineering techniques (PETs) can impact the carbon emission of the Llama 3 model for the code generation task.
• Score: 4.791072577881446
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reducing the environmental impact of AI-based software systems has become critical. The intensive use of large language models (LLMs) in software engineering poses severe challenges regarding computational resources, data centers, and carbon emissions. In this paper, we investigate how prompt engineering techniques (PETs) can impact the carbon emission of the Llama 3 model for the code generation task. We experimented with the CodeXGLUE benchmark to evaluate both energy consumption and the accuracy of the generated code using an isolated testing environment. Our initial results show that the energy consumption of LLMs can be reduced by using specific tags that distinguish different prompt parts. Even though a more in-depth evaluation is needed to confirm our findings, this work suggests that prompt engineering can reduce LLMs' energy consumption during the inference phase without compromising performance, paving the way for further investigations.

Related papers

• Optimizing Large Language Models: Metrics, Energy Efficiency, and Case Study Insights [2.1249213103048414]
  The rapid adoption of large language models (LLMs) has led to significant energy consumption and carbon emissions. This paper explores the integration of energy-efficient optimization techniques in the deployment of LLMs to address these concerns.
  arXiv  Detail & Related papers  (2025-04-07T21:56:59Z)
• Green MLOps to Green GenOps: An Empirical Study of Energy Consumption in Discriminative and Generative AI Operations [2.2765705959685234]
  This study investigates the energy consumption of Discriminative and Generative AI models within real-world MLOps pipelines. We employ software-based power measurements to ensure ease of replication across diverse configurations, models, and datasets.
  arXiv  Detail & Related papers  (2025-03-31T10:28:04Z)
• Unveiling the Energy Vampires: A Methodology for Debugging Software Energy Consumption [5.602876058122268]
  This paper presents an energy debug methodology for identifying and isolating energy consumption hotspots in software systems.<n>Our analysis reveals significant energy consumption differences between Alpine and Ubuntu distributions.<n>By isolating and benchmarking memcpy, we confirm it as the primary cause of the energy discrepancy.
  arXiv  Detail & Related papers  (2024-12-13T11:49:19Z)
• Computing Within Limits: An Empirical Study of Energy Consumption in ML Training and Inference [2.553456266022126]
  Machine learning (ML) has seen tremendous advancements, but its environmental footprint remains a concern. Acknowledging the growing environmental impact of ML this paper investigates Green ML.
  arXiv  Detail & Related papers  (2024-06-20T13:59:34Z)
• Generative AI for Low-Carbon Artificial Intelligence of Things with Large Language Models [67.0243099823109]
  Generative AI (GAI) holds immense potential to reduce carbon emissions of Artificial Intelligence of Things (AIoT) In this article, we explore the potential of GAI for carbon emissions reduction and propose a novel GAI-enabled solution for low-carbon AIoT. We propose a Large Language Model (LLM)-enabled carbon emission optimization framework, in which we design pluggable LLM and Retrieval Augmented Generation (RAG) modules.
  arXiv  Detail & Related papers  (2024-04-28T05:46:28Z)
• Hybrid Heterogeneous Clusters Can Lower the Energy Consumption of LLM Inference Workloads [0.2389598109913753]
  Training and using Large Language Models (LLMs) require large amounts of energy. This paper addresses the challenge of reducing energy consumption in data centers running LLMs. We propose a hybrid data center model that uses a cost-based scheduling framework to dynamically allocate tasks across hardware accelerators.
  arXiv  Detail & Related papers  (2024-04-25T11:24:08Z)
• A Carbon Tracking Model for Federated Learning: Impact of Quantization and Sparsification [5.341266334051207]
  Federated Learning (FL) methods adopt efficient communication technologies to distribute machine learning tasks across edge devices. This paper proposes a framework for real-time monitoring of the energy and carbon footprint impacts of FL systems.
  arXiv  Detail & Related papers  (2023-10-12T07:20:03Z)
• A Comparative Study of Machine Learning Algorithms for Anomaly Detection in Industrial Environments: Performance and Environmental Impact [62.997667081978825]
  This study seeks to address the demands of high-performance machine learning models with environmental sustainability. Traditional machine learning algorithms, such as Decision Trees and Random Forests, demonstrate robust efficiency and performance. However, superior outcomes were obtained with optimised configurations, albeit with a commensurate increase in resource consumption.
  arXiv  Detail & Related papers  (2023-07-01T15:18:00Z)
• Counting Carbon: A Survey of Factors Influencing the Emissions of Machine Learning [77.62876532784759]
  Machine learning (ML) requires using energy to carry out computations during the model training process. The generation of this energy comes with an environmental cost in terms of greenhouse gas emissions, depending on quantity used and the energy source. We present a survey of the carbon emissions of 95 ML models across time and different tasks in natural language processing and computer vision.
  arXiv  Detail & Related papers  (2023-02-16T18:35:00Z)
• Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Model [72.65502770895417]
  We quantify the carbon footprint of BLOOM, a 176-billion parameter language model, across its life cycle. We estimate that BLOOM's final training emitted approximately 24.7 tonnes ofcarboneqif we consider only the dynamic power consumption. We conclude with a discussion regarding the difficulty of precisely estimating the carbon footprint of machine learning models.
  arXiv  Detail & Related papers  (2022-11-03T17:13:48Z)
• Measuring the Carbon Intensity of AI in Cloud Instances [91.28501520271972]
  We provide a framework for measuring software carbon intensity, and propose to measure operational carbon emissions. We evaluate a suite of approaches for reducing emissions on the Microsoft Azure cloud compute platform.
  arXiv  Detail & Related papers  (2022-06-10T17:04:04Z)
• Towards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learning [68.37641996188133]
  We introduce a framework for tracking realtime energy consumption and carbon emissions. We create a leaderboard for energy efficient reinforcement learning algorithms. We propose strategies for mitigation of carbon emissions and reduction of energy consumption.
  arXiv  Detail & Related papers  (2020-01-31T05:12:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.