CodeAlchemy: Synthetic Code Rewriting at Scale

• URL: http://arxiv.org/abs/2606.10087v1
• Date: Mon, 08 Jun 2026 19:15:27 GMT
• Title: CodeAlchemy: Synthetic Code Rewriting at Scale
• Abstract summary: We present CodeAlchemy, a framework that transforms publicly sourced code into semantically-rich training data.<n>We process 3 corpora across 15 languages to generate 500B+ tokens of synthetic data plus 350B reasoning tokens.<n>CodeTrace executes 1.3M+ files across 14 languages and 5K libraries, capturing control flow, state tracking, and library knowledge.
• Score: 18.89851727350116
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Pre-training on raw code teaches syntax but provides sparse signal for diverse real-world task formats. While synthetic data has proven transformative for language models, code remains largely unexplored beyond limited quality improvements. We present CodeAlchemy, a synthetic data generation framework that transforms publicly sourced code into semantically-rich training data through 5 strategies: CodeEnhance (quality-aware rewriting), CodeQA (template-based problems), CodeDev (developer tasks), CodeDialogue (multi-turn conversations), and CodeTrace (execution traces). We process 3 corpora across 15 languages to generate 500B+ tokens of synthetic data plus 350B reasoning tokens, orders of magnitude more than prior efforts. CodeTrace instruments and executes 1.3M+ files across 14 languages and 5K libraries, capturing control flow, state tracking, and library knowledge. We introduce DevEval (developer tasks) and TraceEval (execution prediction) benchmarks; frontier models like Claude Sonnet 4.5 achieve only 5.6% exact match on TraceEval, revealing critical gaps in semantic understanding. Our 3B models achieve 83.5% on HumanEval, 63.2% on MBPP, 8.09% win rate on DevEval, and 15.36 ROUGE-2 on TraceEval, outperforming frontier models 10x the size including 27B Gemma-3 and 32B Granite-4.0.

Related papers

• IFEvalCode: Controlled Code Generation [69.28317223249358]
  The paper introduces forward and backward constraints generation to improve the instruction-following capabilities of Code LLMs.<n>The authors present IFEvalCode, a multilingual benchmark comprising 1.6K test samples across seven programming languages.
  arXiv  Detail & Related papers  (2025-07-30T08:08:48Z)
• UnitCoder: Scalable Iterative Code Synthesis with Unit Test Guidance [65.01483640267885]
  Large Language Models (LLMs) have demonstrated remarkable capabilities in various tasks, yet code generation remains a major challenge.<n>We introduce UnitCoder, a systematic pipeline leveraging model-generated unit tests to guide and validate the code generation process.<n>Our work presents a scalable approach that leverages model-generated unit tests to guide the synthesis of high-quality code data from pre-training corpora.
  arXiv  Detail & Related papers  (2025-02-17T05:37:02Z)
• CodeTree: Agent-guided Tree Search for Code Generation with Large Language Models [106.11371409170818]
  Large language models (LLMs) can act as agents with capabilities to self-refine and improve generated code autonomously. We propose CodeTree, a framework for LLM agents to efficiently explore the search space in different stages of the code generation process. Specifically, we adopted a unified tree structure to explicitly explore different coding strategies, generate corresponding coding solutions, and subsequently refine the solutions.
  arXiv  Detail & Related papers  (2024-11-07T00:09:54Z)
• SemCoder: Training Code Language Models with Comprehensive Semantics Reasoning [24.93484793667691]
  This paper aims to bridge the gap between Code LLMs' reliance on static text data and the need for semantic understanding. We introduce a novel strategy, monologue reasoning, to train Code LLMs to reason comprehensive semantics. We develop SemCoder, a Code LLM with only 6.7B parameters, which shows competitive performance with GPT-3.5-turbo on code generation and execution reasoning tasks.
  arXiv  Detail & Related papers  (2024-06-03T05:36:57Z)
• SOEN-101: Code Generation by Emulating Software Process Models Using Large Language Model Agents [50.82665351100067]
  FlowGen is a code generation framework that emulates software process models based on multiple Large Language Model (LLM) agents. We evaluate FlowGenScrum on four benchmarks: HumanEval, HumanEval-ET, MBPP, and MBPP-ET.
  arXiv  Detail & Related papers  (2024-03-23T14:04:48Z)
• CodeShell Technical Report [23.741490720927068]
  We present CodeShell-Base, a foundation model with 8K context length, showcasing exceptional proficiency in code comprehension. We have curated 100 billion high-quality pre-training data from GitHub. Benefiting from the high-quality data, CodeShell-Base outperforms CodeLlama in Humaneval after training on just 500 billion tokens (5 epochs)
  arXiv  Detail & Related papers  (2024-03-23T07:29:41Z)
• StarCoder 2 and The Stack v2: The Next Generation [105.93298676368798]
  We train StarCoder2 models with 3B, 7B, and 15B parameters on 3.3 to 4.3 trillion tokens. We thoroughly evaluate them on a comprehensive set of Code LLM benchmarks. Our large model, StarCoder2- 15B, significantly outperforms other models of comparable size.
  arXiv  Detail & Related papers  (2024-02-29T13:53:35Z)
• BioCoder: A Benchmark for Bioinformatics Code Generation with Large Language Models [27.772192759716116]
  We present BioCoder, a benchmark developed to evaluate large language models (LLMs) in generating bioinformatics-specific code. BioCoder spans much of the field, covering cross-file dependencies, class declarations, and global variables. We show that the overall coverage of the included code is representative of the full spectrum of bioinformatics calculations.
  arXiv  Detail & Related papers  (2023-08-31T04:52:58Z)
• AI-assisted Code Authoring at Scale: Fine-tuning, deploying, and mixed methods evaluation [9.915327592560896]
  We present CodeCompose, an AI-assisted code authoring tool developed and deployed at Meta internally. CodeCompose is based on the InCoder LLM that merges generative capabilities with bi-directionality. In a random sample of 20K source code files, we are able to reproduce hidden lines between 40% and 58% of the time, an improvement of 1.4x and 4.1x over a model trained only on public data.
  arXiv  Detail & Related papers  (2023-05-20T00:45:15Z)
• CodeExp: Explanatory Code Document Generation [94.43677536210465]
  Existing code-to-text generation models produce only high-level summaries of code. We conduct a human study to identify the criteria for high-quality explanatory docstring for code. We present a multi-stage fine-tuning strategy and baseline models for the task.
  arXiv  Detail & Related papers  (2022-11-25T18:05:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.