COBOL-Coder: Domain-Adapted Large Language Models for COBOL Code Generation and Translation

• URL: http://arxiv.org/abs/2604.03986v1
• Date: Sun, 05 Apr 2026 06:12:49 GMT
• Title: COBOL-Coder: Domain-Adapted Large Language Models for COBOL Code Generation and Translation
• Abstract summary: Existing large language models (LLMs) struggle to generate and translate code correctly.<n>We introduce an automated data curation pipeline that combines compiler-guided validation with multi-stage similarity-based filtering.<n>Code-Coder achieves up to 73.95 percent compilation success rate and 49.33 Pass-1 onEval, compared to 41.8 percent and 16.4 for GPT-4o.
• Score: 10.733306714499067
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: COBOL remains a critical language for mainframe systems, yet existing large language models (LLMs) struggle to generate and translate COBOL code correctly. This paper reports our experience in developing and evaluating domain-adapted LLMs for COBOL and mainframe software engineering. We introduce (1) an automated data curation pipeline that combines compiler-guided validation with multi-stage similarity-based filtering to construct high-quality COBOL training data, and (2) COBOL-Coder, a COBOL-specialized LLM fine-tuned on the curated COBOL domain data. We evaluate COBOL-Coder on two tasks: code generation (on COBOLEval and COBOLCodeBench) and code translation (on COBOL-JavaTrans, our proposed benchmark for bidirectional COBOL-Java translation). In our experiments, COBOL-Coder achieves up to a 73.95 percent compilation success rate and 49.33 Pass-1 on COBOLEval, compared to 41.8 percent and 16.4 for GPT-4o, while most open-source baselines (e.g., CodeGemma, CodeLlama, StarCoder2) fail to produce compilable programs. For Java-to-COBOL translation, COBOL-Coder reaches 34.93 Pass-1, whereas general-purpose LLMs achieve near-zero scores. To assess the usability of LLM-generated code in real-world settings, we conduct a survey with experienced COBOL developers. Participants consistently report that COBOL-Coder exhibits stronger COBOL awareness, has more reliable program structure, and is better aligned with enterprise practices than general-purpose LLMs.

Related papers

• SEDCoT: Enhancing LLM-Based COBOL Code Translation via Symbolic Execution and Delta Debugging [49.898541267264115]
  Maintenance is increasingly challenging due to outdated technologies, sparse documentation, and developer retirement.<n>Traditional rule-based transcompilers yield outputs that are difficult to read and maintain.<n>We propose SEDCoT, a novel-to-C translation framework.
  arXiv  Detail & Related papers  (2026-07-05T03:07:10Z)
• COBOLAssist: Analyzing and Fixing Compilation Errors for LLM-Powered COBOL Code Generation [10.733306714499067]
  This paper investigates the challenges of compilation errors and introduces a framework leveraging large language models (LLMs) to address these issues.<n>We first categorize the common compilation errors in LLM-generated code into three groups: incomplete code errors, syntax errors, and type-related errors.
  arXiv  Detail & Related papers  (2026-04-05T05:51:54Z)
• IFEvalCode: Controlled Code Generation [69.28317223249358]
  The paper introduces forward and backward constraints generation to improve the instruction-following capabilities of Code LLMs.<n>The authors present IFEvalCode, a multilingual benchmark comprising 1.6K test samples across seven programming languages.
  arXiv  Detail & Related papers  (2025-07-30T08:08:48Z)
• Enhancing COBOL Code Explanations: A Multi-Agents Approach Using Large Language Models [1.835291631794229]
  We propose a multi-agent approach that generates explanations for functions, files, and the overall project.<n>We evaluate the effectiveness of our approach using 14 open-source, real-world projects.
  arXiv  Detail & Related papers  (2025-07-02T22:28:35Z)
• Automated Validation of COBOL to Java Transformation [5.162381960073761]
  We propose a framework and a tool to help validate the equivalence of and translated Java.<n>The results can also help repair the code if there are some issues and provide feedback to the AI model to improve.
  arXiv  Detail & Related papers  (2025-04-14T06:13:32Z)
• Automated Testing of COBOL to Java Transformation [5.395038658302793]
  We develop a testing framework for IBM Watsonx Code Assistant for Z (WCA4Z) [5], an industrial tool designed for Java translation.<n>Our framework uses symbolic execution to generate unit tests for mocking external calls and transforming them into JUnit tests to validate semantic equivalence with translated Java.
  arXiv  Detail & Related papers  (2025-04-14T04:53:30Z)
• CRUXEval-X: A Benchmark for Multilingual Code Reasoning, Understanding and Execution [50.1875460416205]
  The CRUXEVAL-X code reasoning benchmark contains 19 programming languages.<n>It comprises at least 600 subjects for each language, along with 19K content-consistent tests in total.<n>Even a model trained solely on Python can achieve at most 34.4% Pass@1 in other languages.
  arXiv  Detail & Related papers  (2024-08-23T11:43:00Z)
• CP-BCS: Binary Code Summarization Guided by Control Flow Graph and Pseudo Code [79.87518649544405]
  We present a control flow graph and pseudo code guided binary code summarization framework called CP-BCS. CP-BCS utilizes a bidirectional instruction-level control flow graph and pseudo code that incorporates expert knowledge to learn the comprehensive binary function execution behavior and logic semantics.
  arXiv  Detail & Related papers  (2023-10-24T14:20:39Z)
• X-COBOL: A Dataset of COBOL Repositories [4.8342038441006805]
  We constructed a dataset of 84 repositories mined from GitHub, containing rich metadata on the development cycle of the projects. Our dataset also provides 1255 files present inside the mined repositories.
  arXiv  Detail & Related papers  (2023-06-08T02:42:09Z)
• ReACC: A Retrieval-Augmented Code Completion Framework [53.49707123661763]
  We propose a retrieval-augmented code completion framework, leveraging both lexical copying and referring to code with similar semantics by retrieval. We evaluate our approach in the code completion task in Python and Java programming languages, achieving a state-of-the-art performance on CodeXGLUE benchmark.
  arXiv  Detail & Related papers  (2022-03-15T08:25:08Z)
• CoSQA: 20,000+ Web Queries for Code Search and Question Answering [63.92224685262063]
  CoSQA dataset includes 20,604 labels for pairs of natural language queries and codes. We introduce a contrastive learning method dubbed CoCLR to enhance query-code matching. We show that evaluated on CodeXGLUE with the same CodeBERT model, training on CoSQA improves the accuracy of code question answering by 5.1%.
  arXiv  Detail & Related papers  (2021-05-27T15:37:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.