AI-Powered, But Power-Hungry? Energy Efficiency of LLM-Generated Code

• URL: http://arxiv.org/abs/2502.02412v1
• Date: Tue, 04 Feb 2025 15:32:34 GMT
• Title: AI-Powered, But Power-Hungry? Energy Efficiency of LLM-Generated Code
• Authors: Lola Solovyeva, Sophie Weidmann, Fernando Castor,
• Abstract summary: This paper presents the first study analyzing the energy efficiency and performance of LLM-generated code for three programming languages Python, Java, and C++. Our results show that the models are much more successful in generating Python and Java than C++ code.
• Score: 45.77395425799378
• License:
• Abstract: Large language models (LLMs) are used in software development to assist in various tasks, e.g., code generation and code completion, but empirical evaluations of the quality of the results produced by these models focus on correctness and ignore other relevant aspects, such as their performance and energy efficiency. Studying the performance of LLM-produced programs is essential to understand how well LLMs can support the construction of performance- and energy-critical software, such as operating systems, servers, and mobile applications. This paper presents the first study analyzing the energy efficiency and performance of LLM-generated code for three programming languages Python, Java, and C++, on two platforms, a Mac and a PC, leveraging three frontier LLMs, Github Copilot, GPT-4o, and the recently-released OpenAI o1-mini, and targeting ``hard'' programming problems from LeetCode. Our results show that the models are much more successful in generating Python and Java than C++ code.

Related papers

• GREEN-CODE: Optimizing Energy Efficiency in Large Language Models for Code Generation [1.5749416770494706]
  This work proposes a framework for energy-aware code generation in Large Language Models (LLMs) We train a Reinforcement Learning (RL) agent that learns to balance the trade-offs between accuracy, latency, and energy consumption. Results show that our method reduces the energy consumption between 23-50 % on average for code generation tasks without significantly affecting accuracy.
  arXiv  Detail & Related papers  (2025-01-19T10:44:03Z)
• PerfCodeGen: Improving Performance of LLM Generated Code with Execution Feedback [78.89596149768458]
  Large Language Models (LLMs) are widely adopted for assisting in software development tasks. We propose PerfCodeGen, a training-free framework that enhances the performance of LLM-generated code.
  arXiv  Detail & Related papers  (2024-11-18T06:22:38Z)
• Crystal: Illuminating LLM Abilities on Language and Code [58.5467653736537]
  We propose a pretraining strategy to enhance the integration of natural language and coding capabilities. The resulting model, Crystal, demonstrates remarkable capabilities in both domains.
  arXiv  Detail & Related papers  (2024-11-06T10:28:46Z)
• Can Large-Language Models Help us Better Understand and Teach the Development of Energy-Efficient Software? [2.8812501020074968]
  Energy-efficient software engineering techniques are often absent from undergraduate curricula. We propose to develop a learning module for energy-efficient software, suitable for incorporation into an undergraduate software engineering class.
  arXiv  Detail & Related papers  (2024-10-30T01:09:32Z)
• Large Language Models for Energy-Efficient Code: Emerging Results and Future Directions [2.848398051763324]
  We propose a novel application of large language models (LLMs) as codes for energy efficiency. We describe and evaluate a prototype, finding that over 6 small programs our system can improve energy efficiency in 3 of them, up to 2x better than compiler optimizations alone.
  arXiv  Detail & Related papers  (2024-10-11T20:35:40Z)
• What's Wrong with Your Code Generated by Large Language Models? An Extensive Study [80.18342600996601]
  Large language models (LLMs) produce code that is shorter yet more complicated as compared to canonical solutions. We develop a taxonomy of bugs for incorrect codes that includes three categories and 12 sub-categories, and analyze the root cause for common bug types. We propose a novel training-free iterative method that introduces self-critique, enabling LLMs to critique and correct their generated code based on bug types and compiler feedback.
  arXiv  Detail & Related papers  (2024-07-08T17:27:17Z)
• Performance-Aligned LLMs for Generating Fast Code [2.180216161965907]
  We introduce a reinforcement learning based methodology to align the outputs of code LLMs with performance. We demonstrate that our fine-tuned model improves the expected speedup of generated code over base models for a set of benchmark tasks.
  arXiv  Detail & Related papers  (2024-04-29T16:52:38Z)
• On Evaluating the Efficiency of Source Code Generated by LLMs [31.8121544062256]
  More efficient code can lead to higher performance and execution efficiency of programs and software completed by LLM-assisted programming. First, we evaluate the efficiency of the code generated by LLMs on two benchmarks, HumanEval and MBPP. Then, we choose a set of programming problems from the online judge platform LeetCode to conduct a more difficult evaluation.
  arXiv  Detail & Related papers  (2024-04-09T05:59:39Z)
• If LLM Is the Wizard, Then Code Is the Wand: A Survey on How Code Empowers Large Language Models to Serve as Intelligent Agents [81.60906807941188]
  Large language models (LLMs) are trained on a combination of natural language and formal language (code) Code translates high-level goals into executable steps, featuring standard syntax, logical consistency, abstraction, and modularity.
  arXiv  Detail & Related papers  (2024-01-01T16:51:20Z)
• LLM-Assisted Code Cleaning For Training Accurate Code Generators [53.087019724256606]
  We investigate data quality for code and find that making the code more structured and readable leads to improved code generation performance of the system. We build a novel data-cleaning pipeline that uses these principles to transform existing programs. We evaluate our approach on two challenging algorithmic code generation benchmarks and find that fine-tuning CodeLLaMa-7B improves the performance by up to 30% compared to fine-tuning on the original dataset.
  arXiv  Detail & Related papers  (2023-11-25T02:45:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.