On the Effectiveness of Training Data Optimization for LLM-based Code Generation: An Empirical Study

• URL: http://arxiv.org/abs/2512.24570v1
• Date: Wed, 31 Dec 2025 02:30:05 GMT
• Title: On the Effectiveness of Training Data Optimization for LLM-based Code Generation: An Empirical Study
• Authors: Shiqi Kuang, Zhao Tian, Tao Xiao, Dong Wang, Junjie Chen,
• Abstract summary: Data synthesis is the most effective technique for improving functional correctness and reducing code smells.<n>Data combined with data achieves the strongest overall performance.<n>This work is a first step toward a systematic understanding of training data optimization and combination strategies.
• Score: 14.089680223493842
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) have achieved remarkable progress in code generation, largely driven by the availability of high-quality code datasets for effective training. To further improve data quality, numerous training data optimization techniques have been proposed; however, their overall effectiveness has not been systematically evaluated. To bridge this gap, we conduct the first large-scale empirical study, examining five widely-used training data optimization techniques and their pairwise combinations for LLM-based code generation across three benchmarks and four LLMs. Our results show that data synthesis is the most effective technique for improving functional correctness and reducing code smells, although it performs relatively worse on code maintainability compared to data refactoring, cleaning, and selection. Regarding combinations, we find that most combinations do not further improve functional correctness but can effectively enhance code quality (code smells and maintainability). Among all combinations, data synthesis combined with data refactoring achieves the strongest overall performance. Furthermore, our fine-grained analysis reinforces these findings and provides deeper insights into how individual techniques and their combinations influence code generation effectiveness. Overall, this work represents a first step toward a systematic understanding of training data optimization and combination strategies, offering practical guidance for future research and deployment in LLM-based code generation.

Related papers

• Afterburner: Reinforcement Learning Facilitates Self-Improving Code Efficiency Optimization [46.33639431414019]
  Large Language Models generate functionally correct solutions but often fall short in code efficiency.<n>We introduce a novel test-time iterative optimization framework to address this.
  arXiv  Detail & Related papers  (2025-05-29T12:14:29Z)
• Enhancing LLM Code Generation: A Systematic Evaluation of Multi-Agent Collaboration and Runtime Debugging for Improved Accuracy, Reliability, and Latency [0.0]
  We study how different programming activities compositions and training paradigms influence code generation effectiveness.<n>Our findings provide valuable insights for organizations seeking robust AI-driven coding solutions.
  arXiv  Detail & Related papers  (2025-05-04T14:44:27Z)
• Integrating Symbolic Execution into the Fine-Tuning of Code-Generating LLMs [1.8838588087156363]
  This paper investigates the fine-tuning of code-generating Large Language Models (LLMs)<n>We enhance the training data for the reward model with the help of symbolic execution techniques.<n>Our reward models, fine-tuned on this dataset, demonstrate significant improvements over the baseline, CodeRL.
  arXiv  Detail & Related papers  (2025-04-21T16:29:07Z)
• Star-Agents: Automatic Data Optimization with LLM Agents for Instruction Tuning [71.2981957820888]
  We propose a novel Star-Agents framework, which automates the enhancement of data quality across datasets. The framework initially generates diverse instruction data with multiple LLM agents through a bespoke sampling method. The generated data undergo a rigorous evaluation using a dual-model method that assesses both difficulty and quality.
  arXiv  Detail & Related papers  (2024-11-21T02:30:53Z)
• OpenCoder: The Open Cookbook for Top-Tier Code Large Language Models [76.59316249991657]
  Large language models (LLMs) for code have become indispensable in various domains, including code generation, reasoning tasks and agent systems.<n>While open-access code LLMs are increasingly approaching the performance levels of proprietary models, high-quality code LLMs remain limited.<n>We introduce OpenCoder, a top-tier code LLM that not only achieves performance comparable to leading models but also serves as an "open cookbook" for the research community.
  arXiv  Detail & Related papers  (2024-11-07T17:47:25Z)
• CodeDPO: Aligning Code Models with Self Generated and Verified Source Code [52.70310361822519]
  We propose CodeDPO, a framework that integrates preference learning into code generation to improve two key code preference factors: code correctness and efficiency.<n>CodeDPO employs a novel dataset construction method, utilizing a self-generation-and-validation mechanism that simultaneously generates and evaluates code and test cases.
  arXiv  Detail & Related papers  (2024-10-08T01:36:15Z)
• Code Less, Align More: Efficient LLM Fine-tuning for Code Generation with Data Pruning [4.975728472540823]
  We present techniques that integrate various clustering and pruning metrics to selectively reduce training data without compromising the accuracy and functionality of the generated code. Our experiments show that these pruning strategies not only reduce the computational resources needed but also enhance the overall quality code generation.
  arXiv  Detail & Related papers  (2024-07-06T10:30:43Z)
• Exploring Data-Efficient Adaptation of Large Language Models for Code Generation [64.5583894165813]
  We propose a novel adaptation approach named DEED, which stands for Data-Efficient adaptation with Error-Driven learning for code generation.<n> Experimental results show that, compared to other mainstream fine-tuning approaches, DEED achieves superior performance with few training data.
  arXiv  Detail & Related papers  (2024-02-29T16:09:02Z)
• Code Needs Comments: Enhancing Code LLMs with Comment Augmentation [91.52444946362547]
  We introduce a novel data augmentation method that generates comments for existing code, coupled with a data filtering strategy that filters out code data poorly correlated with natural language. We conducted experiments on three code-focused Large Language Models and observed consistent improvements in performance on two widely-used programming skill benchmarks.
  arXiv  Detail & Related papers  (2024-02-20T13:56:38Z)
• LLM-Assisted Code Cleaning For Training Accurate Code Generators [53.087019724256606]
  We investigate data quality for code and find that making the code more structured and readable leads to improved code generation performance of the system. We build a novel data-cleaning pipeline that uses these principles to transform existing programs. We evaluate our approach on two challenging algorithmic code generation benchmarks and find that fine-tuning CodeLLaMa-7B improves the performance by up to 30% compared to fine-tuning on the original dataset.
  arXiv  Detail & Related papers  (2023-11-25T02:45:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.