Automated and Context-Aware Code Documentation Leveraging Advanced LLMs

• URL: http://arxiv.org/abs/2509.14273v1
• Date: Tue, 16 Sep 2025 06:27:09 GMT
• Title: Automated and Context-Aware Code Documentation Leveraging Advanced LLMs
• Authors: Swapnil Sharma Sarker, Tanzina Taher Ifty,
• Abstract summary: Existing automated approaches primarily focused on code summarization, leaving a gap in template-based documentation generation.<n>We develop a novel, context-aware dataset for Javadoc generation that includes critical structural and semantic information from modern Javas.<n>We evaluate five open-source LLMs (including LLaMA-3.1, Gemma Phi-3, Mistral, Qwen-2.5) using zero-shot, few-shot, and fine-tuned setups and provide a comparative analysis of their performance.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Code documentation is essential to improve software maintainability and comprehension. The tedious nature of manual code documentation has led to much research on automated documentation generation. Existing automated approaches primarily focused on code summarization, leaving a gap in template-based documentation generation (e.g., Javadoc), particularly with publicly available Large Language Models (LLMs). Furthermore, progress in this area has been hindered by the lack of a Javadoc-specific dataset that incorporates modern language features, provides broad framework/library coverage, and includes necessary contextual information. This study aims to address these gaps by developing a tailored dataset and assessing the capabilities of publicly available LLMs for context-aware, template-based Javadoc generation. In this work, we present a novel, context-aware dataset for Javadoc generation that includes critical structural and semantic information from modern Java codebases. We evaluate five open-source LLMs (including LLaMA-3.1, Gemma-2, Phi-3, Mistral, Qwen-2.5) using zero-shot, few-shot, and fine-tuned setups and provide a comparative analysis of their performance. Our results demonstrate that LLaMA 3.1 performs consistently well and is a reliable candidate for practical, automated Javadoc generation, offering a viable alternative to proprietary systems.

Related papers

• Let's Use ChatGPT To Write Our Paper! Benchmarking LLMs To Write the Introduction of a Research Paper [64.50822834679101]
  SciIG is a task that evaluates LLMs' ability to produce coherent introductions from titles, abstracts, and related works.<n>We assess five state-of-the-art models, including open-source (DeepSeek-v3, Gemma-3-12B, LLaMA 4-Maverick, MistralAI Small 3.1) and closed-source GPT-4o systems.<n>Results demonstrate LLaMA-4 Maverick's superior performance on most metrics, particularly in semantic similarity and faithfulness.
  arXiv  Detail & Related papers  (2025-08-19T21:11:11Z)
• On LLM-Assisted Generation of Smart Contracts from Business Processes [0.08192907805418582]
  Large language models (LLMs) have changed the reality of how software is produced.<n>We present an exploratory study to investigate the use of LLMs for generating smart contract code from business process descriptions.<n>Our results show that LLM performance falls short of the perfect reliability required for smart contract development.
  arXiv  Detail & Related papers  (2025-07-30T20:39:45Z)
• Beyond Isolated Dots: Benchmarking Structured Table Construction as Deep Knowledge Extraction [80.88654868264645]
  Arranged and Organized Extraction Benchmark designed to evaluate ability of large language models to comprehend fragmented documents.<n>AOE includes 11 carefully crafted tasks across three diverse domains, requiring models to generate context-specific schema tailored to varied input queries.<n>Results show that even the most advanced models struggled significantly.
  arXiv  Detail & Related papers  (2025-07-22T06:37:51Z)
• Formal Methods Meets Readability: Auto-Documenting JML Java Code [1.0650780147044159]
  This paper investigates whether formal specifications using Java Modeling Language (JML) can enhance the quality of Large Language Model (LLM)-generated Javadocs.<n>We present a systematic comparison of documentation generated from JML-annotated and non-annotated Java classes.<n>Our findings demonstrate that JML significantly improves class-level documentation, with more moderate gains at the method level.
  arXiv  Detail & Related papers  (2025-06-10T20:39:29Z)
• M-DocSum: Do LVLMs Genuinely Comprehend Interleaved Image-Text in Document Summarization? [49.53982792497275]
  We investigate whether Large Vision-Language Models (LVLMs) genuinely comprehend interleaved image-text in the document.<n>Existing document understanding benchmarks often assess LVLMs using question-answer formats.<n>We introduce a novel and challenging Multimodal Document Summarization Benchmark (M-DocSum-Bench)<n>M-DocSum-Bench comprises 500 high-quality arXiv papers, along with interleaved multimodal summaries aligned with human preferences.
  arXiv  Detail & Related papers  (2025-03-27T07:28:32Z)
• Analysis on LLMs Performance for Code Summarization [0.0]
  Large Language Models (LLMs) have significantly advanced the field of code summarization.<n>This study aims to perform a comparative analysis of several open-source LLMs, namely LLaMA-3, Phi-3, Mistral, and Gemma.
  arXiv  Detail & Related papers  (2024-12-22T17:09:34Z)
• Doc2OracLL: Investigating the Impact of Documentation on LLM-based Test Oracle Generation [0.716879432974126]
  In Java, Javadoc comments provide structured, natural language documentation embedded directly in the source code.<n>We dive deep into investigating the impact of Javadoc comments on test oracle generation (TOG)
  arXiv  Detail & Related papers  (2024-12-12T15:27:47Z)
• Integrating Planning into Single-Turn Long-Form Text Generation [66.08871753377055]
  We propose to use planning to generate long form content. Our main novelty lies in a single auxiliary task that does not require multiple rounds of prompting or planning. Our experiments demonstrate on two datasets from different domains, that LLMs fine-tuned with the auxiliary task generate higher quality documents.
  arXiv  Detail & Related papers  (2024-10-08T17:02:40Z)
• DOCBENCH: A Benchmark for Evaluating LLM-based Document Reading Systems [99.17123445211115]
  We introduce DocBench, a benchmark to evaluate large language model (LLM)-based document reading systems. Our benchmark involves the recruitment of human annotators and the generation of synthetic questions. It includes 229 real documents and 1,102 questions, spanning across five different domains and four major types of questions.
  arXiv  Detail & Related papers  (2024-07-15T13:17:42Z)
• CodeRAG-Bench: Can Retrieval Augment Code Generation? [78.37076502395699]
  We conduct a systematic, large-scale analysis of code generation using retrieval-augmented generation.<n>We first curate a comprehensive evaluation benchmark, CodeRAG-Bench, encompassing three categories of code generation tasks.<n>We examine top-performing models on CodeRAG-Bench by providing contexts retrieved from one or multiple sources.
  arXiv  Detail & Related papers  (2024-06-20T16:59:52Z)
• ML-Bench: Evaluating Large Language Models and Agents for Machine Learning Tasks on Repository-Level Code [76.84199699772903]
  ML-Bench is a benchmark rooted in real-world programming applications that leverage existing code repositories to perform tasks. To evaluate both Large Language Models (LLMs) and AI agents, two setups are employed: ML-LLM-Bench for assessing LLMs' text-to-code conversion within a predefined deployment environment, and ML-Agent-Bench for testing autonomous agents in an end-to-end task execution within a Linux sandbox environment.
  arXiv  Detail & Related papers  (2023-11-16T12:03:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.