Leveraging Metamemory Mechanisms for Enhanced Data-Free Code Generation in LLMs

• URL: http://arxiv.org/abs/2501.07892v1
• Date: Tue, 14 Jan 2025 07:16:43 GMT
• Title: Leveraging Metamemory Mechanisms for Enhanced Data-Free Code Generation in LLMs
• Authors: Shuai Wang, Liang Ding, Yibing Zhan, Yong Luo, Zheng He, Dapeng Tao,
• Abstract summary: M2WF is a framework for improving large language models' one-time code generation. Unlike prior methods, it minimizes dependency on curated data and adapts to various coding scenarios. The code and framework will be publicly available on GitHub and HuggingFace.
• Score: 44.80420740455364
• License:
• Abstract: Automated code generation using large language models (LLMs) has gained attention due to its efficiency and adaptability. However, real-world coding tasks or benchmarks like HumanEval and StudentEval often lack dedicated training datasets, challenging existing few-shot prompting approaches that rely on reference examples. Inspired by human metamemory-a cognitive process involving recall and evaluation-we present a novel framework (namely M^2WF) for improving LLMs' one-time code generation. This approach enables LLMs to autonomously generate, evaluate, and utilize synthetic examples to enhance reliability and performance. Unlike prior methods, it minimizes dependency on curated data and adapts flexibly to various coding scenarios. Our experiments demonstrate significant improvements in coding benchmarks, offering a scalable and robust solution for data-free environments. The code and framework will be publicly available on GitHub and HuggingFace.

Related papers

• Does Few-Shot Learning Help LLM Performance in Code Synthesis? [40.35198206199065]
  This work focuses on the few-shot examples present in most code generation prompts. Our work offers 2 approaches for selecting few-shot examples, a model-free method, CODEEXEMPLAR-FREE, and a model-based method, CODEEXEMPLAR-BASED. Both methods significantly improve CodeLlama's coding ability across the popular HumanEval+ coding benchmark.
  arXiv  Detail & Related papers  (2024-12-03T23:19:40Z)
• Star-Agents: Automatic Data Optimization with LLM Agents for Instruction Tuning [71.2981957820888]
  We propose a novel Star-Agents framework, which automates the enhancement of data quality across datasets. The framework initially generates diverse instruction data with multiple LLM agents through a bespoke sampling method. The generated data undergo a rigorous evaluation using a dual-model method that assesses both difficulty and quality.
  arXiv  Detail & Related papers  (2024-11-21T02:30:53Z)
• FALCON: Feedback-driven Adaptive Long/short-term memory reinforced Coding Optimization system [8.775210512734603]
  Large language models (LLMs) have achieved significant progress in automated code generation. challenges in supervised fine-tuning (SFT) and reinforcement learning from human feedback (RLHF) led to failures in generating precise, human-intent-aligned code. We propose Feedback-driven Adaptive Long/short-term memory reinforced Coding Optimization (i.e., FALCON)
  arXiv  Detail & Related papers  (2024-10-28T12:18:22Z)
• Reference Trustable Decoding: A Training-Free Augmentation Paradigm for Large Language Models [79.41139393080736]
  Large language models (LLMs) have rapidly advanced and demonstrated impressive capabilities. In-Context Learning (ICL) and. Efficient Fine-Tuning (PEFT) are currently two mainstream methods for augmenting. LLMs to downstream tasks. We propose Reference Trustable Decoding (RTD), a paradigm that allows models to quickly adapt to new tasks without fine-tuning.
  arXiv  Detail & Related papers  (2024-09-30T10:48:20Z)
• A Survey on Evaluating Large Language Models in Code Generation Tasks [30.256255254277914]
  This paper provides a comprehensive review of the current methods and metrics used to evaluate the performance of Large Language Models (LLMs) in code generation tasks. With the rapid growth in demand for automated software development, LLMs have demonstrated significant potential in the field of code generation.
  arXiv  Detail & Related papers  (2024-08-29T12:56:06Z)
• SIaM: Self-Improving Code-Assisted Mathematical Reasoning of Large Language Models [54.78329741186446]
  We propose a novel paradigm that uses a code-based critic model to guide steps including question-code data construction, quality control, and complementary evaluation. Experiments across both in-domain and out-of-domain benchmarks in English and Chinese demonstrate the effectiveness of the proposed paradigm.
  arXiv  Detail & Related papers  (2024-08-28T06:33:03Z)
• Code Less, Align More: Efficient LLM Fine-tuning for Code Generation with Data Pruning [4.975728472540823]
  We present techniques that integrate various clustering and pruning metrics to selectively reduce training data without compromising the accuracy and functionality of the generated code. Our experiments show that these pruning strategies not only reduce the computational resources needed but also enhance the overall quality code generation.
  arXiv  Detail & Related papers  (2024-07-06T10:30:43Z)
• One Token Can Help! Learning Scalable and Pluggable Virtual Tokens for Retrieval-Augmented Large Language Models [67.49462724595445]
  Retrieval-augmented generation (RAG) is a promising way to improve large language models (LLMs) We propose a novel method that involves learning scalable and pluggable virtual tokens for RAG.
  arXiv  Detail & Related papers  (2024-05-30T03:44:54Z)
• Online Adaptation of Language Models with a Memory of Amortized Contexts [82.02369596879817]
  Memory of Amortized Contexts (MAC) is an efficient and effective online adaptation framework for large language models. We show how MAC can be combined with and improve the performance of popular alternatives such as retrieval augmented generations.
  arXiv  Detail & Related papers  (2024-03-07T08:34:57Z)
• SEED: Customize Large Language Models with Sample-Efficient Adaptation for Code Generation [35.88318116340547]
  We propose a novel adaptation approach named SEED, which stands for Sample-Efficient adaptation with Error-Driven learning for code generation. We show that SEED achieves superior performance with few training samples, showing an average relative improvement of 54.7% in Pass@1 on multiple code generation benchmarks.
  arXiv  Detail & Related papers  (2024-02-29T16:09:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.