Towards Understanding the Capability of Large Language Models on Code Clone Detection: A Survey

• URL: http://arxiv.org/abs/2308.01191v3
• Date: Sun, 6 Aug 2023 01:40:59 GMT
• Title: Towards Understanding the Capability of Large Language Models on Code Clone Detection: A Survey
• Authors: Shihan Dou, Junjie Shan, Haoxiang Jia, Wenhao Deng, Zhiheng Xi, Wei He, Yueming Wu, Tao Gui, Yang Liu, Xuanjing Huang
• Abstract summary: Large language models (LLMs) possess diverse code-related knowledge, making them versatile for various software engineering challenges. This paper provides the first comprehensive evaluation of LLMs for clone detection, covering different clone types, languages, and prompts. We find advanced LLMs excel in detecting complex semantic clones, surpassing existing methods.
• Score: 40.99060616674878
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Code cloning, the duplication of code fragments, is common in software development. While some reuse aids productivity, excessive cloning hurts maintainability and introduces bugs. Hence, automatic code clone detection is vital. Meanwhile, large language models (LLMs) possess diverse code-related knowledge, making them versatile for various software engineering challenges. However, LLMs' performance in code clone detection is unclear and needs more study for accurate assessment. In this paper, we provide the first comprehensive evaluation of LLMs for clone detection, covering different clone types, languages, and prompts. We find advanced LLMs excel in detecting complex semantic clones, surpassing existing methods. Adding intermediate reasoning steps via chain-of-thought prompts noticeably enhances performance. Additionally, representing code as vector embeddings, especially with text encoders, effectively aids clone detection.Lastly, the ability of LLMs to detect code clones differs among various programming languages. Our study suggests that LLMs have potential for clone detection due to their language capabilities, offering insights for developing robust LLM-based methods to enhance software engineering.

Related papers

• Development and Benchmarking of Multilingual Code Clone Detector [2.253851493296371]
  multilingual code clone detectors make it easier to add new language support by providing syntax information of the target language only. We propose a multilingual code block extraction method based on ANTLR generation and implement a multilingual code clone detector (MSCCD) Compared to ten state-of-the-art detectors, MSCCD performs at an average level while it also supports a significantly larger number of languages.
  arXiv  Detail & Related papers  (2024-09-10T03:08:33Z)
• Large Language Models for cross-language code clone detection [3.5202378300682162]
  Cross-lingual code clone detection has gained traction with the software engineering community. Inspired by the significant advances in machine learning, this paper revisits cross-lingual code clone detection.
  arXiv  Detail & Related papers  (2024-08-08T12:57:14Z)
• What's Wrong with Your Code Generated by Large Language Models? An Extensive Study [80.18342600996601]
  Large language models (LLMs) produce code that is shorter yet more complicated as compared to canonical solutions. We develop a taxonomy of bugs for incorrect codes that includes three categories and 12 sub-categories, and analyze the root cause for common bug types. We propose a novel training-free iterative method that introduces self-critique, enabling LLMs to critique and correct their generated code based on bug types and compiler feedback.
  arXiv  Detail & Related papers  (2024-07-08T17:27:17Z)
• Assessing the Code Clone Detection Capability of Large Language Models [0.0]
  The evaluation involves testing the models on a variety of code pairs of different clone types and levels of similarity. Findings indicate that GPT-4 consistently surpasses GPT-3.5 across all clone types.
  arXiv  Detail & Related papers  (2024-07-02T16:20:44Z)
• Exploring Multi-Lingual Bias of Large Code Models in Code Generation [55.336629780101475]
  Code generation aims to synthesize code and fulfill functional requirements based on natural language (NL) specifications. Despite the effectiveness, we observe a noticeable multilingual bias in the generation performance of large code models (LCMs) LCMs demonstrate proficiency in generating solutions when provided with instructions in English, yet may falter when faced with semantically equivalent instructions in other NLs such as Chinese.
  arXiv  Detail & Related papers  (2024-04-30T08:51:49Z)
• CodeIP: A Grammar-Guided Multi-Bit Watermark for Large Language Models of Code [56.019447113206006]
  Large Language Models (LLMs) have achieved remarkable progress in code generation. CodeIP is a novel multi-bit watermarking technique that embeds additional information to preserve provenance details. Experiments conducted on a real-world dataset across five programming languages demonstrate the effectiveness of CodeIP.
  arXiv  Detail & Related papers  (2024-04-24T04:25:04Z)
• If LLM Is the Wizard, Then Code Is the Wand: A Survey on How Code Empowers Large Language Models to Serve as Intelligent Agents [81.60906807941188]
  Large language models (LLMs) are trained on a combination of natural language and formal language (code) Code translates high-level goals into executable steps, featuring standard syntax, logical consistency, abstraction, and modularity.
  arXiv  Detail & Related papers  (2024-01-01T16:51:20Z)
• AdaCCD: Adaptive Semantic Contrasts Discovery Based Cross Lingual Adaptation for Code Clone Detection [69.79627042058048]
  AdaCCD is a novel cross-lingual adaptation method that can detect cloned codes in a new language without annotations in that language. We evaluate the cross-lingual adaptation results of AdaCCD by constructing a multilingual code clone detection benchmark consisting of 5 programming languages.
  arXiv  Detail & Related papers  (2023-11-13T12:20:48Z)
• Testing LLMs on Code Generation with Varying Levels of Prompt Specificity [0.0]
  Large language models (LLMs) have demonstrated unparalleled prowess in mimicking human-like text generation and processing. The potential to transform natural language prompts into executable code promises a major shift in software development practices.
  arXiv  Detail & Related papers  (2023-11-10T23:41:41Z)
• Evaluation of Contrastive Learning with Various Code Representations for Code Clone Detection [3.699097874146491]
  We evaluate contrastive learning for detecting semantic clones of code snippets. We use CodeTransformator to create a dataset that mimics plagiarised code based on competitive programming solutions. The results of our evaluation show that proposed models perform diversely in each task, however the performance of the graph-based models is generally above the others.
  arXiv  Detail & Related papers  (2022-06-17T12:25:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.