A test-free semantic mistakes localization framework in Neural Code Translation

• URL: http://arxiv.org/abs/2410.22818v1
• Date: Wed, 30 Oct 2024 08:53:33 GMT
• Title: A test-free semantic mistakes localization framework in Neural Code Translation
• Authors: Lei Chen, Sai Zhang, Fangzhou Xu, Zhenchang Xing, Liang Wan, Xiaowang Zhang, Zhiyong Feng,
• Abstract summary: We present EISP, a static analysis framework based on the Large Language Model (LLM) The framework generates a semantic mapping between source code and translated code. EISP connects each pair of sub-code fragments with fine-grained knowledge hints through an AI chain.
• Score: 32.5036379897325
• License:
• Abstract: In the task of code translation, neural network-based models have been shown to frequently produce semantically erroneous code that deviates from the original logic of the source code. This issue persists even with advanced large models. Although a recent approach proposed using test cases to identify these semantic errors, it relies heavily on the quality of the test cases and is not applicable to code snippets without test cases in real-world scenarios. Therefore, We present EISP, a static analysis framework based on the Large Language Model (LLM).First, the framework generates a semantic mapping between source code and translated code. Next, each sub-code fragment is identified by recursively traversing the abstract syntax tree of the source code, and its corresponding translated code fragment is found through the semantic mapping. Finally, EISP connects each pair of sub-code fragments with fine-grained knowledge hints through an AI chain to assist LLMs in discovering semantic mistakes in the translated code. In our benchmark evaluation, the EISP framework, based on GPT-4o mini, achieved an accuracy of 82.3\%, representing a 20.3\% improvement over baseline methods using the same base model, and a 7.4\% improvement compared to dynamic analysis methods that require test cases and manual intervention. To our knowledge, EISP is the first tool to locate semantic errors in translated code without test cases or compilable code. This innovative tool provides the software engineering community with a new way to deal with code fragments without test cases.

Related papers

• Fix the Tests: Augmenting LLMs to Repair Test Cases with Static Collector and Neural Reranker [9.428021853841296]
  We propose SYNTER, a novel approach to automatically repair obsolete test cases via precise and concise TROCtxs construction. With the augmentation of constructed TROCtxs, hallucinations are reduced by 57.1%.
  arXiv  Detail & Related papers  (2024-07-04T04:24:43Z)
• Uncovering LLM-Generated Code: A Zero-Shot Synthetic Code Detector via Code Rewriting [78.48355455324688]
  We propose a novel zero-shot synthetic code detector based on the similarity between the code and its rewritten variants. Our results demonstrate a notable enhancement over existing synthetic content detectors designed for general texts.
  arXiv  Detail & Related papers  (2024-05-25T08:57:28Z)
• Source Code Vulnerability Detection: Combining Code Language Models and Code Property Graphs [5.953617559607503]
  Vul-LMGNN is a unified model that combines pre-trained code language models with code property graphs. Vul-LMGNN constructs a code property graph that integrates various code attributes into a unified graph structure. To effectively retain dependency information among various attributes, we introduce a gated code Graph Neural Network.
  arXiv  Detail & Related papers  (2024-04-23T03:48:18Z)
• Neural Models for Source Code Synthesis and Completion [0.0]
  Natural language (NL) to code suggestion systems assist developers in Integrated Development Environments (IDEs) by translating NL utterances into compilable code snippet. Current approaches mainly involve hard-coded, rule-based systems based on semantic parsing. We present sequence-to-sequence deep learning models and training paradigms to map NL to general-purpose programming languages.
  arXiv  Detail & Related papers  (2024-02-08T17:10:12Z)
• Probing Semantic Grounding in Language Models of Code with Representational Similarity Analysis [0.11470070927586018]
  We propose using Representational Similarity Analysis to probe the semantic grounding in language models of code. We probe representations from the CodeBERT model for semantic grounding by using the data from the IBM CodeNet dataset. Our experiments with semantic perturbations in code reveal that CodeBERT is able to robustly distinguish between semantically correct and incorrect code.
  arXiv  Detail & Related papers  (2022-07-15T19:04:43Z)
• Enhancing Semantic Code Search with Multimodal Contrastive Learning and Soft Data Augmentation [50.14232079160476]
  We propose a new approach with multimodal contrastive learning and soft data augmentation for code search. We conduct extensive experiments to evaluate the effectiveness of our approach on a large-scale dataset with six programming languages.
  arXiv  Detail & Related papers  (2022-04-07T08:49:27Z)
• Software Vulnerability Detection via Deep Learning over Disaggregated Code Graph Representation [57.92972327649165]
  This work explores a deep learning approach to automatically learn the insecure patterns from code corpora. Because code naturally admits graph structures with parsing, we develop a novel graph neural network (GNN) to exploit both the semantic context and structural regularity of a program.
  arXiv  Detail & Related papers  (2021-09-07T21:24:36Z)
• Multimodal Representation for Neural Code Search [18.371048875103497]
  We introduce tree-serialization methods on a simplified form of AST and build the multimodal representation for the code data. Our results show that both our tree-serialized representations and multimodal learning model improve the performance of neural code search.
  arXiv  Detail & Related papers  (2021-07-02T12:08:19Z)
• Zero-Shot Cross-lingual Semantic Parsing [56.95036511882921]
  We study cross-lingual semantic parsing as a zero-shot problem without parallel data for 7 test languages. We propose a multi-task encoder-decoder model to transfer parsing knowledge to additional languages using only English-Logical form paired data. Our system frames zero-shot parsing as a latent-space alignment problem and finds that pre-trained models can be improved to generate logical forms with minimal cross-lingual transfer penalty.
  arXiv  Detail & Related papers  (2021-04-15T16:08:43Z)
• Deep Graph Matching and Searching for Semantic Code Retrieval [76.51445515611469]
  We propose an end-to-end deep graph matching and searching model based on graph neural networks. We first represent both natural language query texts and programming language code snippets with the unified graph-structured data. In particular, DGMS not only captures more structural information for individual query texts or code snippets but also learns the fine-grained similarity between them.
  arXiv  Detail & Related papers  (2020-10-24T14:16:50Z)
• Contrastive Code Representation Learning [95.86686147053958]
  We show that the popular reconstruction-based BERT model is sensitive to source code edits, even when the edits preserve semantics. We propose ContraCode: a contrastive pre-training task that learns code functionality, not form.
  arXiv  Detail & Related papers  (2020-07-09T17:59:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.