Generating High-Quality Datasets for Code Editing via Open-Source Language Models

• URL: http://arxiv.org/abs/2509.25203v3
• Date: Tue, 07 Oct 2025 07:44:59 GMT
• Title: Generating High-Quality Datasets for Code Editing via Open-Source Language Models
• Abstract summary: We introduce OpenCodeEdit, an open-source pipeline that synthesizes realistic code-edit triplets.<n>The pipeline produces both concise "lazy" instructions and more detailed "descriptive" ones, and applies filtering based on diffs and topics to guarantee data quality and variety.<n>Fine-tuning three advanced base models on OCEDataFT leads to significant performance boosts on the CanItEdit benchmark, with relative pass@1 improvements ranging from 4.50% to 20.79%.
• Score: 49.01026966779838
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Code editing plays a vital role in software engineering, requiring developers to adjust existing code according to natural language instructions while keeping functionality intact and avoiding unnecessary modifications. However, commit-based datasets commonly used for this task are often noisy, lack diversity, and fail to reflect the style of real-world edit instructions. To address this, we introduce OpenCodeEdit, an open-source pipeline that leverages multiple LLMs to synthesize realistic code-edit triplets. The pipeline produces both concise "lazy" instructions and more detailed "descriptive" ones, and applies filtering based on diffs and topics to guarantee data quality and variety. Using this process, we construct OCEDataFT, a curated dataset of 20K samples. Fine-tuning three advanced base models on OCEDataFT leads to significant performance boosts on the CanItEdit benchmark, with relative pass@1 improvements ranging from 4.50% to 20.79%. Notably, the resulting models achieve performance close to closed-source systems, narrowing the gap to GPT-4 to just 3.54%, without relying on proprietary resources or manual annotation.

Related papers

• Industrial-Instruction: An End-to-End Framework for Building Instruction-Tuning and Benchmark Datasets from Industrial Technical Reports [1.806188080472605]
  Industrial technical reports contain high-value knowledge for maintenance, troubleshooting, and product engineering.<n>No public instruction-tuning or benchmark datasets are built from such documents.<n>We contribute two open QA datasets built from real industrial technical reports and the end-to-end pipeline that produces them.
  arXiv  Detail & Related papers  (2026-08-24T05:28:25Z)
• To Diff or Not to Diff? Structure-Aware and Adaptive Output Formats for Efficient LLM-based Code Editing [54.399335188667045]
  BlockDiff and FuncDiff are structure-aware diff formats that represent changes as block-level rewrites of syntactically coherent units.<n>AdaEdit is a general adaptive edit strategy that trains LLMs to dynamically choose the most token-efficient format between a given diff format and full code.
  arXiv  Detail & Related papers  (2026-04-30T01:14:13Z)
• Leveraging LLMs for Multi-File DSL Code Generation: An Industrial Case Study [39.936784974007644]
  Large language models (LLMs) perform strongly on general-purpose code generation, yet their applicability to enterprise domain-specific languages (s) remains underexplored.<n>We report an industrial case study at BMW that adapts code-oriented LLMs to generate and modify project-root DSL artifacts for an Xtext-based DSL.<n>We encode as structured, path-response generation at repository scale and learning cross-file dependencies.
  arXiv  Detail & Related papers  (2026-04-27T16:38:01Z)
• Envisioning Future Interactive Web Development: Editing Webpage with Natural Language [5.799436684542269]
  We introduce a novel, automated data generation pipeline that uses Large Language Models to synthesize a high-quality fine-tuning dataset for web editing.<n>By fine-tuning models on Instruct4Edit, we demonstrate consistent improvement in translating human intent into precise, structurally coherent, and visually accurate code changes.
  arXiv  Detail & Related papers  (2025-10-30T14:09:50Z)
• Easy Dataset: A Unified and Extensible Framework for Synthesizing LLM Fine-Tuning Data from Unstructured Documents [34.55648227035844]
  Easy dataset is a framework for synthesizing fine-tuning data from unstructured documents.<n>It allows users to configure text extraction models and chunking strategies to transform raw documents into coherent text chunks.<n>It then leverages a persona-driven prompting approach to generate diverse question-answer pairs.<n>Experiments on a financial question-answering task show that fine-tuning LLMs on the synthesized dataset significantly improves domain-specific performance.
  arXiv  Detail & Related papers  (2025-07-05T11:38:59Z)
• RefEdit: A Benchmark and Method for Improving Instruction-based Image Editing Model on Referring Expressions [56.9437856499838]
  We introduce RefEdit -- an instruction-based editing model trained on our scalable synthetic data generation pipeline.<n>Our RefEdit, trained on only 20,000 editing triplets, outperforms the Flux/SD3 model-based baselines trained on millions of data.
  arXiv  Detail & Related papers  (2025-06-03T23:20:24Z)
• OpenCodeInstruct: A Large-scale Instruction Tuning Dataset for Code LLMs [62.68905180014956]
  We introduce OpenCodeInstruct, the largest open-access instruction tuning dataset, comprising 5 million diverse samples.<n>Each sample includes a programming question, solution, test cases, execution feedback, and LLM-generated quality assessments.<n>We fine-tune various base models, including LLaMA and Qwen, across multiple scales (1B+, 3B+, and 7B+) using our dataset.
  arXiv  Detail & Related papers  (2025-04-05T02:52:16Z)
• Bridging the Editing Gap in LLMs: FineEdit for Precise and Targeted Text Modifications [4.751608548909266]
  FineEdit is a specialized editing model explicitly trained for context-aware text modifications.<n>FineEdit outperforms state-of-the-art models on single-turn edits, up to 30% over Llama-3.2-3B, and exceeding Mistral-7B-OpenOrca performance by over 40% on direct editing tasks.
  arXiv  Detail & Related papers  (2025-02-19T01:41:44Z)
• OpenCoder: The Open Cookbook for Top-Tier Code Large Language Models [76.59316249991657]
  Large language models (LLMs) for code have become indispensable in various domains, including code generation, reasoning tasks and agent systems.<n>While open-access code LLMs are increasingly approaching the performance levels of proprietary models, high-quality code LLMs remain limited.<n>We introduce OpenCoder, a top-tier code LLM that not only achieves performance comparable to leading models but also serves as an "open cookbook" for the research community.
  arXiv  Detail & Related papers  (2024-11-07T17:47:25Z)
• Better Synthetic Data by Retrieving and Transforming Existing Datasets [63.875064274379824]
  We introduce DataTune, a method to make better use of publicly available datasets to improve automatic dataset generation. On a diverse set of language-based tasks, we find that finetuning language models via DataTune improves over a few-shot prompting baseline by 49%. We find that dataset transformation significantly increases the diversity and difficulty of generated data on many tasks.
  arXiv  Detail & Related papers  (2024-04-22T17:15:32Z)
• LLM-Assisted Code Cleaning For Training Accurate Code Generators [53.087019724256606]
  We investigate data quality for code and find that making the code more structured and readable leads to improved code generation performance of the system. We build a novel data-cleaning pipeline that uses these principles to transform existing programs. We evaluate our approach on two challenging algorithmic code generation benchmarks and find that fine-tuning CodeLLaMa-7B improves the performance by up to 30% compared to fine-tuning on the original dataset.
  arXiv  Detail & Related papers  (2023-11-25T02:45:50Z)
• InstructCoder: Instruction Tuning Large Language Models for Code Editing [26.160498475809266]
  We explore the use of Large Language Models (LLMs) to edit code based on user instructions. InstructCoder is the first instruction-tuning dataset designed to adapt LLMs for general-purpose code editing. Our findings reveal that open-source LLMs fine-tuned on InstructCoder can significantly enhance the accuracy of code edits.
  arXiv  Detail & Related papers  (2023-10-31T10:15:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.