Effi-Code: Unleashing Code Efficiency in Language Models

• URL: http://arxiv.org/abs/2410.10209v2
• Date: Sat, 19 Oct 2024 12:39:11 GMT
• Title: Effi-Code: Unleashing Code Efficiency in Language Models
• Authors: Dong Huang, Guangtao Zeng, Jianbo Dai, Meng Luo, Han Weng, Yuhao Qing, Heming Cui, Zhijiang Guo, Jie M. Zhang,
• Abstract summary: Effi-Code is an approach to enhancing code generation in large language models. Effi-Code offers a scalable and generalizable approach to improving code generation in AI systems.
• Score: 17.355845751737423
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As the use of large language models (LLMs) for code generation becomes more prevalent in software development, it is critical to enhance both the efficiency and correctness of the generated code. Existing methods and models primarily focus on the correctness of LLM-generated code, ignoring efficiency. In this work, we present Effi-Code, an approach to enhancing code generation in LLMs that can improve both efficiency and correctness. We introduce a Self-Optimization process based on Overhead Profiling that leverages open-source LLMs to generate a high-quality dataset of correct and efficient code samples. This dataset is then used to fine-tune various LLMs. Our method involves the iterative refinement of generated code, guided by runtime performance metrics and correctness checks. Extensive experiments demonstrate that models fine-tuned on the Effi-Code show significant improvements in both code correctness and efficiency across task types. For example, the pass@1 of DeepSeek-Coder-6.7B-Instruct generated code increases from \textbf{43.3\%} to \textbf{76.8\%}, and the average execution time for the same correct tasks decreases by \textbf{30.5\%}. Effi-Code offers a scalable and generalizable approach to improving code generation in AI systems, with potential applications in software development, algorithm design, and computational problem-solving. The source code of Effi-Code was released in \url{https://github.com/huangd1999/Effi-Code}.

Related papers

• LLM4EFFI: Leveraging Large Language Models to Enhance Code Efficiency and Correctness [38.399282089600284]
  Large Language Models (LLMs) have demonstrated impressive performance in code generation. tool: ulineLarge ulineLanguage ulineModel for Code ulineEfficiency is a novel framework that enables LLMs to generate code that balances both efficiency and correctness.
  arXiv  Detail & Related papers  (2025-02-17T07:01:18Z)
• COFFE: A Code Efficiency Benchmark for Code Generation [20.79578698298569]
  We propose COFFE, a code generation benchmark for evaluating the time efficiency of LLM-generated code solutions. COFFE contains 398 and 358 problems for function-level and file-level code generation, respectively. For the time evaluation metric, we propose efficienct@k based on CPU instruction count to ensure a stable and solid comparison between different solutions.
  arXiv  Detail & Related papers  (2025-02-05T02:08:51Z)
• PerfCodeGen: Improving Performance of LLM Generated Code with Execution Feedback [78.89596149768458]
  Large Language Models (LLMs) are widely adopted for assisting in software development tasks. We propose PerfCodeGen, a training-free framework that enhances the performance of LLM-generated code.
  arXiv  Detail & Related papers  (2024-11-18T06:22:38Z)
• Crystal: Illuminating LLM Abilities on Language and Code [58.5467653736537]
  We propose a pretraining strategy to enhance the integration of natural language and coding capabilities. The resulting model, Crystal, demonstrates remarkable capabilities in both domains.
  arXiv  Detail & Related papers  (2024-11-06T10:28:46Z)
• Rethinking Code Refinement: Learning to Judge Code Efficiency [60.04718679054704]
  Large Language Models (LLMs) have demonstrated impressive capabilities in understanding and generating codes. We propose a novel method based on the code language model that is trained to judge the efficiency between two different codes. We validate our method on multiple programming languages with multiple refinement steps, demonstrating that the proposed method can effectively distinguish between more and less efficient versions of code.
  arXiv  Detail & Related papers  (2024-10-29T06:17:37Z)
• CodeDPO: Aligning Code Models with Self Generated and Verified Source Code [52.70310361822519]
  We propose CodeDPO, a framework that integrates preference learning into code generation to improve two key code preference factors: code correctness and efficiency. CodeDPO employs a novel dataset construction method, utilizing a self-generation-and-validation mechanism that simultaneously generates and evaluates code and test cases.
  arXiv  Detail & Related papers  (2024-10-08T01:36:15Z)
• ECCO: Can We Improve Model-Generated Code Efficiency Without Sacrificing Functional Correctness? [12.862825053595934]
  ECCO is a benchmark for evaluating program efficiency via two paradigms: natural language (NL) based code generation and history-based code editing. We find that adding execution information often helps maintain functional correctness, and NL feedback enhances more on efficiency.
  arXiv  Detail & Related papers  (2024-07-19T05:47:40Z)
• Code-Optimise: Self-Generated Preference Data for Correctness and Efficiency [15.593172556501704]
  We introduce Code-Optimise, a framework that incorporates both correctness (passed, failed) and runtime as learning signals. Our framework is both lightweight and robust as it dynamically selects solutions to reduce overfitting. As a by-product, the average length of the generated solutions is reduced by up to 48% on MBPP and 23% on HumanEval.
  arXiv  Detail & Related papers  (2024-06-18T11:05:37Z)
• How Efficient is LLM-Generated Code? A Rigorous & High-Standard Benchmark [39.13045037676502]
  Development of large language models (LLMs) has significantly pushed the frontiers of program synthesis. Most evaluation frameworks focus on the (functional) correctness of generated code; efficiency, as an important measure of code quality, has been overlooked in existing evaluations. We develop ENAMEL, a rigorous and high-standard benchmark for evaluating the capability of LLMs in generating efficient code.
  arXiv  Detail & Related papers  (2024-06-10T04:19:20Z)
• EffiLearner: Enhancing Efficiency of Generated Code via Self-Optimization [16.238980494717303]
  Large language models (LLMs) have shown remarkable progress in code generation, but their generated code often suffers from inefficiency. We propose textbfEffiLearner, a self-optimization framework that utilizes execution overhead profiles. EffiLearner first generates code using an LLM, then executes it locally to capture execution time and memory usage profiles.
  arXiv  Detail & Related papers  (2024-05-24T03:48:15Z)
• EffiBench: Benchmarking the Efficiency of Automatically Generated Code [16.19693502619949]
  EffiBench is a benchmark with 1,000 efficiency-critical coding problems. Each problem is paired with an executable human-written canonical solution. We empirically examine the ability of 42 large language models to generate efficient code.
  arXiv  Detail & Related papers  (2024-02-03T05:24:39Z)
• StepCoder: Improve Code Generation with Reinforcement Learning from Compiler Feedback [58.20547418182074]
  We introduce StepCoder, a novel framework for code generation, consisting of two main components. CCCS addresses the exploration challenge by breaking the long sequences code generation task into a Curriculum of Code Completion Subtasks. FGO only optimize the model by masking the unexecuted code segments to provide Fine-Grained Optimization. Our method improves the ability to explore the output space and outperforms state-of-the-art approaches in corresponding benchmarks.
  arXiv  Detail & Related papers  (2024-02-02T13:14:31Z)
• LLM-Assisted Code Cleaning For Training Accurate Code Generators [53.087019724256606]
  We investigate data quality for code and find that making the code more structured and readable leads to improved code generation performance of the system. We build a novel data-cleaning pipeline that uses these principles to transform existing programs. We evaluate our approach on two challenging algorithmic code generation benchmarks and find that fine-tuning CodeLLaMa-7B improves the performance by up to 30% compared to fine-tuning on the original dataset.
  arXiv  Detail & Related papers  (2023-11-25T02:45:50Z)
• CodeFuse-13B: A Pretrained Multi-lingual Code Large Language Model [58.127534002232096]
  This paper introduces CodeFuse-13B, an open-sourced pre-trained code LLM. It is specifically designed for code-related tasks with both English and Chinese prompts. CodeFuse achieves its effectiveness by utilizing a high quality pre-training dataset.
  arXiv  Detail & Related papers  (2023-10-10T02:38:44Z)
• LongCoder: A Long-Range Pre-trained Language Model for Code Completion [56.813974784131624]
  LongCoder employs a sliding window mechanism for self-attention and introduces two types of globally accessible tokens. Bridge tokens are inserted throughout the input sequence to aggregate local information and facilitate global interaction. memory tokens are included to highlight important statements that may be invoked later and need to be memorized.
  arXiv  Detail & Related papers  (2023-06-26T17:59:24Z)
• Highly Parallel Autoregressive Entity Linking with Discriminative Correction [51.947280241185]
  We propose a very efficient approach that parallelizes autoregressive linking across all potential mentions. Our model is >70 times faster and more accurate than the previous generative method.
  arXiv  Detail & Related papers  (2021-09-08T17:28:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.