Enhancing COBOL Code Explanations: A Multi-Agents Approach Using Large Language Models

• URL: http://arxiv.org/abs/2507.02182v1
• Date: Wed, 02 Jul 2025 22:28:35 GMT
• Title: Enhancing COBOL Code Explanations: A Multi-Agents Approach Using Large Language Models
• Abstract summary: We propose a multi-agent approach that generates explanations for functions, files, and the overall project.<n>We evaluate the effectiveness of our approach using 14 open-source, real-world projects.
• Score: 1.835291631794229
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Common Business Oriented Language (COBOL) is a programming language used to develop business applications that are widely adopted by financial, business, and government agencies. Due to its age, complexity, and declining number of COBOL developers, maintaining COBOL codebases is becoming increasingly challenging. In particular, the lack of documentation makes it difficult for new developers to effectively understand and maintain COBOL systems. Existing research utilizes large language models (LLMs) to explain the functionality of code snippets. However, COBOL presents unique challenges due to its architectural and syntactical differences, which often cause its code to exceed the token window size of LLMs. In this work, we propose a multi-agent approach that leverages two LLM-based agents working collaboratively to generate explanations for functions, files, and the overall project. These agents incorporate together by utilizing contextual information from the codebase into the code explanation prompts. We evaluate the effectiveness of our approach using 14 open-source, real-world COBOL projects. Our results indicate that our approach performs significantly better than the baseline in function code explanation, with improvements of 12.67%, 18.59%, and 0.62% in terms of METEOR, chrF, and SentenceBERT scores, respectively. At the file level, our approach effectively explains both short and long COBOL files that exceed the token window size of LLMs and surpass the baseline by 4.21%, 10.72%, and 14.68% in explaining the purpose, functionality, and clarity of the generated explanation. At the project level, our approach generates explanations that convey the functionality and purpose of 82% of the selected projects.

Related papers

• SEDCoT: Enhancing LLM-Based COBOL Code Translation via Symbolic Execution and Delta Debugging [49.898541267264115]
  Maintenance is increasingly challenging due to outdated technologies, sparse documentation, and developer retirement.<n>Traditional rule-based transcompilers yield outputs that are difficult to read and maintain.<n>We propose SEDCoT, a novel-to-C translation framework.
  arXiv  Detail & Related papers  (2026-07-05T03:07:10Z)
• COBOL-Coder: Domain-Adapted Large Language Models for COBOL Code Generation and Translation [10.733306714499067]
  Existing large language models (LLMs) struggle to generate and translate code correctly.<n>We introduce an automated data curation pipeline that combines compiler-guided validation with multi-stage similarity-based filtering.<n>Code-Coder achieves up to 73.95 percent compilation success rate and 49.33 Pass-1 onEval, compared to 41.8 percent and 16.4 for GPT-4o.
  arXiv  Detail & Related papers  (2026-04-05T06:12:49Z)
• COBOLAssist: Analyzing and Fixing Compilation Errors for LLM-Powered COBOL Code Generation [10.733306714499067]
  This paper investigates the challenges of compilation errors and introduces a framework leveraging large language models (LLMs) to address these issues.<n>We first categorize the common compilation errors in LLM-generated code into three groups: incomplete code errors, syntax errors, and type-related errors.
  arXiv  Detail & Related papers  (2026-04-05T05:51:54Z)
• IFEvalCode: Controlled Code Generation [69.28317223249358]
  The paper introduces forward and backward constraints generation to improve the instruction-following capabilities of Code LLMs.<n>The authors present IFEvalCode, a multilingual benchmark comprising 1.6K test samples across seven programming languages.
  arXiv  Detail & Related papers  (2025-07-30T08:08:48Z)
• An Empirical Study on the Effectiveness of Large Language Models for Binary Code Understanding [50.17907898478795]
  This work proposes a benchmark to evaluate the effectiveness of Large Language Models (LLMs) in real-world reverse engineering scenarios.<n>Our evaluations reveal that existing LLMs can understand binary code to a certain extent, thereby improving the efficiency of binary code analysis.
  arXiv  Detail & Related papers  (2025-04-30T17:02:06Z)
• Resource-Efficient & Effective Code Summarization [3.512140256677132]
  GreenAI techniques, such as QLoRA, offer a promising path for dealing with large models' sustainability.<n>Our study evaluates two state-of-the-art CLMs across two programming languages: Python and Java.<n>Results show that QLoRA enables efficient fine-tuning of CLMs for code summarization.
  arXiv  Detail & Related papers  (2025-02-05T21:06:30Z)
• Crystal: Illuminating LLM Abilities on Language and Code [58.5467653736537]
  We propose a pretraining strategy to enhance the integration of natural language and coding capabilities. The resulting model, Crystal, demonstrates remarkable capabilities in both domains.
  arXiv  Detail & Related papers  (2024-11-06T10:28:46Z)
• What can Large Language Models Capture about Code Functional Equivalence? [24.178831487657945]
  We introduce SeqCoBench, a benchmark for assessing how Code-LLMs can capture code functional equivalence.<n>We conduct evaluations on state-of-the-art (Code)-LLMs to see if they can discern semantically equivalent or different pairs of programs in SeqCoBench.
  arXiv  Detail & Related papers  (2024-08-20T11:19:06Z)
• CodeGRAG: Bridging the Gap between Natural Language and Programming Language via Graphical Retrieval Augmented Generation [58.84212778960507]
  CodeGRAG builds the graphical view of code blocks based on the control flow and data flow of them to better interpret the programming domain knowledge.<n>CodeGRAG significantly improves the code generation ability of LLMs and can even offer performance gain for cross-lingual code generation.
  arXiv  Detail & Related papers  (2024-05-03T02:48:55Z)
• How Far Have We Gone in Binary Code Understanding Using Large Language Models [51.527805834378974]
  We propose a benchmark to evaluate the effectiveness of Large Language Models (LLMs) in binary code understanding. Our evaluations reveal that existing LLMs can understand binary code to a certain extent, thereby improving the efficiency of binary code analysis.
  arXiv  Detail & Related papers  (2024-04-15T14:44:08Z)
• If LLM Is the Wizard, Then Code Is the Wand: A Survey on How Code Empowers Large Language Models to Serve as Intelligent Agents [81.60906807941188]
  Large language models (LLMs) are trained on a combination of natural language and formal language (code) Code translates high-level goals into executable steps, featuring standard syntax, logical consistency, abstraction, and modularity.
  arXiv  Detail & Related papers  (2024-01-01T16:51:20Z)
• ML-Bench: Evaluating Large Language Models and Agents for Machine Learning Tasks on Repository-Level Code [76.84199699772903]
  ML-Bench is a benchmark rooted in real-world programming applications that leverage existing code repositories to perform tasks. To evaluate both Large Language Models (LLMs) and AI agents, two setups are employed: ML-LLM-Bench for assessing LLMs' text-to-code conversion within a predefined deployment environment, and ML-Agent-Bench for testing autonomous agents in an end-to-end task execution within a Linux sandbox environment.
  arXiv  Detail & Related papers  (2023-11-16T12:03:21Z)
• LongCoder: A Long-Range Pre-trained Language Model for Code Completion [56.813974784131624]
  LongCoder employs a sliding window mechanism for self-attention and introduces two types of globally accessible tokens. Bridge tokens are inserted throughout the input sequence to aggregate local information and facilitate global interaction. memory tokens are included to highlight important statements that may be invoked later and need to be memorized.
  arXiv  Detail & Related papers  (2023-06-26T17:59:24Z)
• X-COBOL: A Dataset of COBOL Repositories [4.8342038441006805]
  We constructed a dataset of 84 repositories mined from GitHub, containing rich metadata on the development cycle of the projects. Our dataset also provides 1255 files present inside the mined repositories.
  arXiv  Detail & Related papers  (2023-06-08T02:42:09Z)
• AstBERT: Enabling Language Model for Code Understanding with Abstract Syntax Tree [3.1087379479634927]
  We propose the AstBERT model, a pre-trained language model aiming to better understand the programming language (PL) using the abstract syntax tree (AST) Specifically, we collect a colossal amount of source codes (both java and python) from GitHub, in which information of the source codes can be interpreted and integrated. Experiment results show that our AstBERT model achieves state-of-the-art performance on both downstream tasks.
  arXiv  Detail & Related papers  (2022-01-20T03:27:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.