Prompt engineering and its implications on the energy consumption of Large Language Models

• URL: http://arxiv.org/abs/2501.05899v1
• Date: Fri, 10 Jan 2025 11:49:31 GMT
• Title: Prompt engineering and its implications on the energy consumption of Large Language Models
• Authors: Riccardo Rubei, Aicha Moussaid, Claudio di Sipio, Davide di Ruscio,
• Abstract summary: Large language models (LLMs) in software engineering pose severe challenges regarding computational resources, data centers, and carbon emissions. In this paper, we investigate how prompt engineering techniques (PETs) can impact the carbon emission of the Llama 3 model for the code generation task.
• Score: 4.791072577881446
• License:
• Abstract: Reducing the environmental impact of AI-based software systems has become critical. The intensive use of large language models (LLMs) in software engineering poses severe challenges regarding computational resources, data centers, and carbon emissions. In this paper, we investigate how prompt engineering techniques (PETs) can impact the carbon emission of the Llama 3 model for the code generation task. We experimented with the CodeXGLUE benchmark to evaluate both energy consumption and the accuracy of the generated code using an isolated testing environment. Our initial results show that the energy consumption of LLMs can be reduced by using specific tags that distinguish different prompt parts. Even though a more in-depth evaluation is needed to confirm our findings, this work suggests that prompt engineering can reduce LLMs' energy consumption during the inference phase without compromising performance, paving the way for further investigations.

Related papers

• Unveiling the Energy Vampires: A Methodology for Debugging Software Energy Consumption [5.602876058122268]
  This paper presents an energy debug methodology for identifying and isolating energy consumption hotspots in software systems. Our analysis reveals significant energy consumption differences between Alpine and Ubuntu distributions. By isolating and benchmarking memcpy, we confirm it as the primary cause of the energy discrepancy.
  arXiv  Detail & Related papers  (2024-12-13T11:49:19Z)
• TinyML NLP Approach for Semantic Wireless Sentiment Classification [49.801175302937246]
  We introduce split learning (SL) as an energy-efficient alternative, privacy-preserving tiny machine learning (MLTiny) scheme. Our results show that SL reduces processing power and CO2 emissions while maintaining high accuracy, whereas FL offers a balanced compromise between efficiency and privacy.
  arXiv  Detail & Related papers  (2024-11-09T21:26:59Z)
• Computing Within Limits: An Empirical Study of Energy Consumption in ML Training and Inference [2.553456266022126]
  Machine learning (ML) has seen tremendous advancements, but its environmental footprint remains a concern. Acknowledging the growing environmental impact of ML this paper investigates Green ML.
  arXiv  Detail & Related papers  (2024-06-20T13:59:34Z)
• Generative AI for Low-Carbon Artificial Intelligence of Things with Large Language Models [67.0243099823109]
  Generative AI (GAI) holds immense potential to reduce carbon emissions of Artificial Intelligence of Things (AIoT) In this article, we explore the potential of GAI for carbon emissions reduction and propose a novel GAI-enabled solution for low-carbon AIoT. We propose a Large Language Model (LLM)-enabled carbon emission optimization framework, in which we design pluggable LLM and Retrieval Augmented Generation (RAG) modules.
  arXiv  Detail & Related papers  (2024-04-28T05:46:28Z)
• Hybrid Heterogeneous Clusters Can Lower the Energy Consumption of LLM Inference Workloads [0.2389598109913753]
  Training and using Large Language Models (LLMs) require large amounts of energy. This paper addresses the challenge of reducing energy consumption in data centers running LLMs. We propose a hybrid data center model that uses a cost-based scheduling framework to dynamically allocate tasks across hardware accelerators.
  arXiv  Detail & Related papers  (2024-04-25T11:24:08Z)
• A Carbon Tracking Model for Federated Learning: Impact of Quantization and Sparsification [5.341266334051207]
  Federated Learning (FL) methods adopt efficient communication technologies to distribute machine learning tasks across edge devices. This paper proposes a framework for real-time monitoring of the energy and carbon footprint impacts of FL systems.
  arXiv  Detail & Related papers  (2023-10-12T07:20:03Z)
• Enhancing Energy-Awareness in Deep Learning through Fine-Grained Energy Measurement [11.37120215795946]
  This paper introduces FECoM (Fine-grained Energy Consumption Meter), a framework for fine-grained Deep Learning energy consumption measurement. FECoM addresses the challenges of measuring energy consumption at fine-grained level by using static instrumentation and considering various factors, including computational load stability and temperature.
  arXiv  Detail & Related papers  (2023-08-23T17:32:06Z)
• A Comparative Study of Machine Learning Algorithms for Anomaly Detection in Industrial Environments: Performance and Environmental Impact [62.997667081978825]
  This study seeks to address the demands of high-performance machine learning models with environmental sustainability. Traditional machine learning algorithms, such as Decision Trees and Random Forests, demonstrate robust efficiency and performance. However, superior outcomes were obtained with optimised configurations, albeit with a commensurate increase in resource consumption.
  arXiv  Detail & Related papers  (2023-07-01T15:18:00Z)
• Counting Carbon: A Survey of Factors Influencing the Emissions of Machine Learning [77.62876532784759]
  Machine learning (ML) requires using energy to carry out computations during the model training process. The generation of this energy comes with an environmental cost in terms of greenhouse gas emissions, depending on quantity used and the energy source. We present a survey of the carbon emissions of 95 ML models across time and different tasks in natural language processing and computer vision.
  arXiv  Detail & Related papers  (2023-02-16T18:35:00Z)
• Measuring the Carbon Intensity of AI in Cloud Instances [91.28501520271972]
  We provide a framework for measuring software carbon intensity, and propose to measure operational carbon emissions. We evaluate a suite of approaches for reducing emissions on the Microsoft Azure cloud compute platform.
  arXiv  Detail & Related papers  (2022-06-10T17:04:04Z)
• Towards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learning [68.37641996188133]
  We introduce a framework for tracking realtime energy consumption and carbon emissions. We create a leaderboard for energy efficient reinforcement learning algorithms. We propose strategies for mitigation of carbon emissions and reduction of energy consumption.
  arXiv  Detail & Related papers  (2020-01-31T05:12:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.