An Extensive Empirical Study on Code Translation Technique

• URL: http://arxiv.org/abs/2608.20776v1
• Date: Fri, 21 Aug 2026 06:41:04 GMT
• Title: An Extensive Empirical Study on Code Translation Technique
• Abstract summary: We compare representative code translation techniques across methodological paradigms and translation granularities.<n>Our analysis considers executable correctness, code similarity, translation direction, translation, and failure patterns.<n>These findings provide empirical evidence and practical guidance for developing more robust, type-aware, structure-aware, and context-aware code translation techniques.
• Score: 3.398993044674732
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Automated code translation is increasingly important for software evolution, yet the relative strengths and limitations of learning-based and large language model (LLM)-based techniques remain insufficiently understood. To address this gap, we conduct a large-scale empirical study comparing representative code translation techniques across methodological paradigms and translation granularities. We evaluate learning-based methods, LLM-based methods, and general-purpose LLMs on multilingual method-level and class-level benchmarks involving multiple programming languages. Our analysis considers executable correctness, code similarity, translation direction, translation granularity, and failure patterns. The results show that LLMs and LLM-based methods generally outperform learning-based methods in method-level correctness, although similarity metrics alone do not reliably reflect functional correctness. Translation direction substantially affects performance, particularly when translating between languages with different type-system characteristics. Class-level translation remains considerably more difficult than method-level translation because it requires preserving global semantics, interfaces, member relationships, and cross-method dependencies. Our error analysis further shows that static semantic errors and logical errors are the primary challenges in existing code translation systems. These findings provide empirical evidence and practical guidance for developing more robust, type-aware, structure-aware, and context-aware code translation techniques.

Related papers

• Large Language Models for Multi-Lingual Equivalent Mutant Detection: An Extended Empirical Study [8.372242332246508]
  Large language models (LLMs) have recently demonstrated remarkable performance across diverse code-related tasks by better capturing program semantics.<n>This paper presents the first comprehensive empirical study on LLMs for, using 3, Java and 1,088 C mutant pairs to benchmark against state-of-the-art methods.<n>LLMs achieve higher F1-scores than the evaluated traditional methods, with fine-tuned code embedding yielding the highest detection accuracy.
  arXiv  Detail & Related papers  (2026-07-01T06:46:13Z)
• Reasoning over Grammar: Can Synthetic Linguistic Reasoning Traces Enhance Low-Resource Machine Translation? [49.7935995447581]
  We investigate whether low-resource machine translation can benefit from structured intermediate steps of linguistic analysis and grammatical reasoning.<n>We propose a pipeline for automatically generating step-by-step linguistic reasoning traces from Universal Dependencies treebanks, dictionaries, and grammar-rule banks.<n>Our results show that linguistic reasoning traces are most effective as inference-time guidance.
  arXiv  Detail & Related papers  (2026-06-02T15:36:12Z)
• Beyond BLEU: A Semantic Evaluation Method for Code Translation [2.3802148866231057]
  We propose a novel evaluation methodology for code translation tasks, emphasizing semantic equivalence over surface-level string similarity.<n>We introduce a semantic correctness score, defined as the proportion of translations that produce correct execution outcomes.<n>Our findings show that LLM-based approaches significantly outperform ones, while BLEU scores show negligible correlation with semantic correctness.
  arXiv  Detail & Related papers  (2026-05-06T17:14:33Z)
• LLMs Explain't: A Post-Mortem on Semantic Interpretability in Transformer Models [3.7965260744113163]
  Large Language Models (LLMs) are becoming increasingly popular in pervasive computing due to their versatility and strong performance.<n>This paper investigates how linguistic abstraction emerges in LLMs, aiming to detect it across different modules.<n> Attention-based explanations collapsed once we tested the core assumption that later-layer representations still correspond to tokens.<n>Property-inference methods applied to embeddings also failed because their high predictive scores were driven by methodological artifacts and dataset structure.
  arXiv  Detail & Related papers  (2026-01-30T12:46:37Z)
• CLaS-Bench: A Cross-Lingual Alignment and Steering Benchmark [21.574271160875046]
  We introduce CLaS-Bench, a benchmark for evaluating language-forcing behavior in large language models (LLMs) across 32 languages.<n>We find that across languages simple residual-based DiffMean method consistently outperforms all other methods.
  arXiv  Detail & Related papers  (2026-01-13T08:42:03Z)
• Code-Switching In-Context Learning for Cross-Lingual Transfer of Large Language Models [64.54005959758733]
  We introduce code-switching in-context learning (CSICL) as a principled and robust approach for overcoming the translation barrier during inference.<n>We conduct extensive experiments across 4 LLMs, 6 datasets, and 10 languages, spanning both knowledge-intensive and reasoning-oriented domains.<n>Our results demonstrate CSICL consistently outperforms X-ICL baselines, achieving gains of 3.1%p and 1.9%p in both target and unseen languages.
  arXiv  Detail & Related papers  (2025-10-07T08:35:42Z)
• Understanding and Mitigating Errors of LLM-Generated RTL Code [7.747889860813149]
  Large language model (LLM) based register-transfer-level (RTL) code generation is promising but the overall success rate remains unsatisfactory.<n>We conduct a comprehensive error analysis and manual categorization.<n>Our findings reveal that most errors stem from insufficient RTL programming knowledge, poor understanding of circuit concepts, or misinterpretation of complex multimodal inputs.
  arXiv  Detail & Related papers  (2025-08-07T11:02:32Z)
• Function-to-Style Guidance of LLMs for Code Translation [59.487054943812836]
  We propose F2STrans, a function-to-style guiding paradigm designed to improve the performance of large language models in code translation.<n>Our approach comprises two key stages: (1) Functional learning, which optimize translation correctness using high-quality source-target code pairs.<n>We introduce a novel code translation benchmark that includes up-to-date source code, extensive test cases, and manually annotated ground-truth translations.
  arXiv  Detail & Related papers  (2025-07-15T08:25:02Z)
• LayerCake: Token-Aware Contrastive Decoding within Large Language Model Layers [53.43862310647276]
  Large language models (LLMs) excel at natural language understanding and generation but remain vulnerable to factual errors.<n>We introduce a token-aware, layer-localized contrastive decoding method that aligns specific token types with their most influential transformer layers to improve factual generation.<n>Our method requires no additional training or model modification, and experiments demonstrate that our method consistently improves factuality across multiple LLMs and various benchmarks.
  arXiv  Detail & Related papers  (2025-07-06T14:35:43Z)
• RustRepoTrans: Repository-level Code Translation Benchmark Targeting Rust [50.65321080814249]
  RustRepoTrans is the first repository-level context code translation benchmark targeting incremental translation.<n>We evaluate seven representative LLMs, analyzing their errors to assess limitations in complex translation scenarios.
  arXiv  Detail & Related papers  (2024-11-21T10:00:52Z)
• LLM-based Translation Inference with Iterative Bilingual Understanding [52.46978502902928]
  We propose a novel Iterative Bilingual Understanding Translation method based on the cross-lingual capabilities of large language models (LLMs)<n>The cross-lingual capability of LLMs enables the generation of contextual understanding for both the source and target languages separately.<n>The proposed IBUT outperforms several strong comparison methods.
  arXiv  Detail & Related papers  (2024-10-16T13:21:46Z)
• A Thorough Examination of Decoding Methods in the Era of LLMs [72.65956436513241]
  Decoding methods play an indispensable role in converting language models from next-token predictors into practical task solvers. This paper provides a comprehensive and multifaceted analysis of various decoding methods within the context of large language models. Our findings reveal that decoding method performance is notably task-dependent and influenced by factors such as alignment, model size, and quantization.
  arXiv  Detail & Related papers  (2024-02-10T11:14:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.