Code2Doc: A Quality-First Curated Dataset for Code Documentation

• URL: http://arxiv.org/abs/2512.18748v2
• Date: Wed, 24 Dec 2025 06:47:27 GMT
• Title: Code2Doc: A Quality-First Curated Dataset for Code Documentation
• Abstract summary: We introduce Code2Doc, a quality-first curated dataset for function-level code documentation generation.<n>Code2Doc consists of 13,358 high-quality function-documentation pairs extracted from open-source repositories.<n>Starting from 52,069 extracted candidates, only 25.6% satisfy all quality constraints.<n>Overall, 86.9% of samples contain explicit type annotations, and only 2.9% are flagged as potentially AI generated.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The performance of automatic code documentation generation models depends critically on the quality of the training data used for supervision. However, most existing code documentation datasets are constructed through large scale scraping of public repositories with limited quality control. As a result, they often contain noisy documentation, extensive duplication, and increasing contamination from AI generated content. These issues weaken the supervision signal available to learning-based models and complicate evaluation. We introduce Code2Doc, a quality-first curated dataset for function-level code documentation generation. Code2Doc consists of 13,358 high-quality function-documentation pairs extracted from widely used open-source repositories spanning five programming languages: Python, Java, TypeScript, JavaScript, and C++. The dataset is constructed using a four-stage curation pipeline that enforces documentation completeness and clarity, filters functions based on structural and complexity criteria, removes exact and near-duplicate code, and identifies documentation likely to be AI generated. Starting from 52,069 extracted candidates, only 25.6% satisfy all quality constraints. We provide a detailed analysis of the resulting dataset, which achieves a mean documentation quality score of 6.93 out of 10. Overall, 86.9% of samples contain explicit type annotations, and only 2.9% are flagged as potentially AI generated. Baseline experiments show that fine-tuning a large language model on Code2Doc yields relative improvements of 29.47% in BLEU and 24.04% in ROUGE-L over zero shot performance, despite the modest dataset size. We release both the dataset and the full curation pipeline to support reproducible research on automatic code documentation generation.

Related papers

• DocAtlas: Multilingual Document Understanding Across 80+ Languages [58.715440331861295]
  We introduce DocAtlas, a framework that constructs high-fidelity OCR datasets and benchmarks covering 82 languages.<n>Our dual pipelines, differential rendering of native DOCX documents and synthetic-based generation for right-to-left scripts produce precise structural annotations.
  arXiv  Detail & Related papers  (2026-05-12T18:09:38Z)
• Integrating Code Metrics into Automated Documentation Generation for Computational Notebooks [0.18665975431697424]
  This paper investigates the role of source code metrics as auxiliary signals for automated documentation generation.<n>It focuses on computational notebooks, a popular medium among data scientists that integrates code, narrative, and results but suffers from inconsistent documentation.<n>Results show that incorporating code metrics improves the accuracy and contextual relevance of generated documentation.
  arXiv  Detail & Related papers  (2026-02-08T21:40:57Z)
• Generating High-Quality Datasets for Code Editing via Open-Source Language Models [49.01026966779838]
  We introduce OpenCodeEdit, an open-source pipeline that synthesizes realistic code-edit triplets.<n>The pipeline produces both concise "lazy" instructions and more detailed "descriptive" ones, and applies filtering based on diffs and topics to guarantee data quality and variety.<n>Fine-tuning three advanced base models on OCEDataFT leads to significant performance boosts on the CanItEdit benchmark, with relative pass@1 improvements ranging from 4.50% to 20.79%.
  arXiv  Detail & Related papers  (2025-09-19T03:57:39Z)
• Docopilot: Improving Multimodal Models for Document-Level Understanding [87.60020625241178]
  We present a high-quality document-level dataset, Doc-750K, designed to support in-depth understanding of multimodal documents.<n>This dataset includes diverse document structures, extensive cross-page dependencies, and real question-answer pairs derived from the original documents.<n>Building on the dataset, we develop a native multimodal model, Docopilot, which can accurately handle document-level dependencies without relying on RAG.
  arXiv  Detail & Related papers  (2025-07-19T16:03:34Z)
• UnitCoder: Scalable Iterative Code Synthesis with Unit Test Guidance [65.01483640267885]
  Large Language Models (LLMs) have demonstrated remarkable capabilities in various tasks, yet code generation remains a major challenge.<n>We introduce UnitCoder, a systematic pipeline leveraging model-generated unit tests to guide and validate the code generation process.<n>Our work presents a scalable approach that leverages model-generated unit tests to guide the synthesis of high-quality code data from pre-training corpora.
  arXiv  Detail & Related papers  (2025-02-17T05:37:02Z)
• EpiCoder: Encompassing Diversity and Complexity in Code Generation [66.43738008739555]
  Existing methods for code generation use code snippets as seed data.<n>We introduce a novel feature tree-based synthesis framework, which revolves around hierarchical code features.<n>Our framework provides precise control over the complexity of the generated code, enabling functionalities that range from function-level operations to multi-file scenarios.
  arXiv  Detail & Related papers  (2025-01-08T18:58:15Z)
• BigDocs: An Open Dataset for Training Multimodal Models on Document and Code Tasks [57.589795399265945]
  We introduce BigDocs-7.5M, a high-quality, open-access dataset comprising 7.5 million multimodal documents across 30 tasks.<n>We also introduce BigDocs-Bench, a benchmark suite with 10 novel tasks.<n>Our experiments show that training with BigDocs-Bench improves average performance up to 25.8% over closed-source GPT-4o.
  arXiv  Detail & Related papers  (2024-12-05T21:41:20Z)
• CodeInsight: A Curated Dataset of Practical Coding Solutions from Stack Overflow [10.19019476978683]
  dataset provides examples that include a clarified intent, code snippets associated, and an average of three related unit tests. Comprising 3,409 crafted examples by Python experts, our dataset is designed for both model finetuning and standalone evaluation.
  arXiv  Detail & Related papers  (2024-09-25T11:18:52Z)
• Using Large Language Models to Enrich the Documentation of Datasets for Machine Learning [1.8270184406083445]
  We explore using large language models (LLM) and prompting strategies to automatically extract dimensions from documents. Our approach could aid data publishers and practitioners in creating machine-readable documentation. We have released an open-source tool implementing our approach and a replication package, including the experiments' code and results.
  arXiv  Detail & Related papers  (2024-04-04T10:09:28Z)
• RepoAgent: An LLM-Powered Open-Source Framework for Repository-level Code Documentation Generation [79.83270415843857]
  We introduce RepoAgent, a large language model powered open-source framework aimed at proactively generating, maintaining, and updating code documentation. We have validated the effectiveness of our approach, showing that RepoAgent excels in generating high-quality repository-level documentation.
  arXiv  Detail & Related papers  (2024-02-26T15:39:52Z)
• CodeExp: Explanatory Code Document Generation [94.43677536210465]
  Existing code-to-text generation models produce only high-level summaries of code. We conduct a human study to identify the criteria for high-quality explanatory docstring for code. We present a multi-stage fine-tuning strategy and baseline models for the task.
  arXiv  Detail & Related papers  (2022-11-25T18:05:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.