Enhancing LLMs in Long Code Translation through Instrumentation and Program State Alignment

• URL: http://arxiv.org/abs/2504.02017v1
• Date: Wed, 02 Apr 2025 13:55:29 GMT
• Title: Enhancing LLMs in Long Code Translation through Instrumentation and Program State Alignment
• Authors: Li Xin-Ye, Du Ya-Li, Li Ming,
• Abstract summary: Code translation aims to transform code between programming languages while preserving functionality.<n>Recent advances in Large Language Models (LLMs) have improved code translation, but challenges remain.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Code translation aims to transform code between programming languages while preserving functionality, with applications in cross-platform development and software migration. Recent advances in Large Language Models (LLMs) have improved code translation, but challenges remain, particularly in inferring program functionality. These issues worsen with longer and more complex code, where current LLMs struggle to handle length and intricate semantics. To evaluate LLMs on long code translation, we introduce LongTrans, a large-scale execution-based benchmark with C++, Java, and Python programs, ranging from hundreds to thousands of tokens. Our empirical study of 12 LLMs reveals a sharp performance decline as code length increases, with even the best-performing model, GPT-4o, achieving only 57.51% computational accuracy. This highlights the need for further research in long code translation. We argue that code translation should maintain invariant functionality while transforming syntax and keywords across languages. Despite differences in appearance, program states should remain consistent throughout execution. To address this, we propose PAST (Program State Alignment augmented Translation), which integrates instrumentation to capture and align program states during translation. This approach is the first to leverage LLMs to insert instrumentation in both original and translated code, tracing program states at runtime. By prompting the LLM to correct errors based on output traces, we mitigate inconsistencies and enhance translation accuracy. Experimental results show significant improvements, with computational accuracy rising from 57.51% to 84.70% for GPT-4o, 50.68% to 69.97% for Mistral-Large-2, and 52.45% to 76.43% for DeepSeek-Coder-V2. These improvements are consistent across models and datasets, with ablation studies confirming the benefits of instrumentation and state alignment.

Related papers

• ObscuraCoder: Powering Efficient Code LM Pre-Training Via Obfuscation Grounding [60.37988508851391]
  Language models (LMs) have become a staple of the code-writing toolbox.<n>Research exploring modifications to Code-LMs' pre-training objectives, geared towards improving data efficiency and better disentangling between syntax and semantics, has been noticeably sparse.<n>In this work, we examine grounding on obfuscated code as a means of helping Code-LMs look beyond the surface-form syntax and enhance their pre-training sample efficiency.
  arXiv  Detail & Related papers  (2025-03-27T23:08:53Z)
• Scalable, Validated Code Translation of Entire Projects using Large Language Models [13.059046327936393]
  Large language models (LLMs) show promise in code translation due to their ability to generate idiomatic code.<n>Existing works have shown a drop in translation success rates for code exceeding around 100 lines.<n>We develop a modular approach to translation, where we partition the code into small code fragments which can be independently translated.<n>We show that we can consistently generate reliable Rust for projects up to 6,600 lines of code and 369 functions, with an average of 73% of functions successfully validated for I/O equivalence.
  arXiv  Detail & Related papers  (2024-12-11T02:31:46Z)
• Unraveling the Potential of Large Language Models in Code Translation: How Far Are We? [4.616570111453259]
  Large language models (LLMs) exhibit state-of-the-art performance in various tasks, but struggle for code translation. We conduct a large-scale empirical study to exploit the capabilities and incapabilities of LLMs in code translation tasks. We propose two methods: (1) intermediary translation which selects an intermediary language between the source and target ones; and (2) self-training which fine-tunes LLMs on self-generated parallel data.
  arXiv  Detail & Related papers  (2024-10-13T12:20:12Z)
• Multilingual Contrastive Decoding via Language-Agnostic Layers Skipping [60.458273797431836]
  Decoding by contrasting layers (DoLa) is designed to improve the generation quality of large language models. We find that this approach does not work well on non-English tasks. Inspired by previous interpretability work on language transition during the model's forward pass, we propose an improved contrastive decoding algorithm.
  arXiv  Detail & Related papers  (2024-07-15T15:14:01Z)
• Towards Translating Real-World Code with LLMs: A Study of Translating to Rust [13.743967357458287]
  Large language models (LLMs) show promise in code translation due to their ability to write code in most programming languages. We conduct our study on code extracted from real-world open source projects. FLOURINE is an end-to-end code translation tool that uses differential fuzzing to check if a Rust translation is I/O equivalent to the original source program.
  arXiv  Detail & Related papers  (2024-05-19T10:54:03Z)
• Exploring and Unleashing the Power of Large Language Models in Automated Code Translation [40.25727029618665]
  This paper investigates diverse LLMs and learning-based transpilers for automated code translation tasks. UniTrans is a Unified code Translation framework, applicable to various LLMs. Three recent LLMs of diverse sizes are tested with UniTrans, and all achieve substantial improvements.
  arXiv  Detail & Related papers  (2024-04-23T00:49:46Z)
• Building Accurate Translation-Tailored LLMs with Language Aware Instruction Tuning [57.323716555996114]
  Off-target translation remains an unsolved problem, especially for low-resource languages. Recent works have either designed advanced prompting strategies to highlight the functionality of translation instructions or exploited the in-context learning ability of LLMs. In this work, we design a two-stage fine-tuning algorithm to improve the instruction-following ability (especially the translation direction) of LLMs.
  arXiv  Detail & Related papers  (2024-03-21T13:47:40Z)
• Speech Translation with Large Language Models: An Industrial Practice [64.5419534101104]
  We introduce LLM-ST, a novel and effective speech translation model constructed upon a pre-trained large language model (LLM) By integrating the large language model (LLM) with a speech encoder and employing multi-task instruction tuning, LLM-ST can produce accurate timestamped transcriptions and translations. Through rigorous experimentation on English and Chinese datasets, we showcase the exceptional performance of LLM-ST.
  arXiv  Detail & Related papers  (2023-12-21T05:32:49Z)
• CodeFuse-13B: A Pretrained Multi-lingual Code Large Language Model [58.127534002232096]
  This paper introduces CodeFuse-13B, an open-sourced pre-trained code LLM. It is specifically designed for code-related tasks with both English and Chinese prompts. CodeFuse achieves its effectiveness by utilizing a high quality pre-training dataset.
  arXiv  Detail & Related papers  (2023-10-10T02:38:44Z)
• Lost in Translation: A Study of Bugs Introduced by Large Language Models while Translating Code [5.915447908295047]
  We present a large-scale empirical study to investigate the ability of general LLMs and code LLMs for code translation. Our study involves the translation of 1,700 code samples from three benchmarks and two real-world projects. We find that correct translations range from 2.1% to 47.3% for the studied LLMs.
  arXiv  Detail & Related papers  (2023-08-06T13:33:13Z)
• LEVER: Learning to Verify Language-to-Code Generation with Execution [64.36459105535]
  We propose LEVER, a simple approach to improve language-to-code generation by learning to verify the generated programs with their execution results. Specifically, we train verifiers to determine whether a program sampled from the LLMs is correct or not based on the natural language input, the program itself and its execution results. LEVER consistently improves over the base code LLMs(4.6% to 10.9% with code-davinci) and achieves new state-of-the-art results on all of them.
  arXiv  Detail & Related papers  (2023-02-16T18:23:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.