GenCode: A Generic Data Augmentation Framework for Boosting Deep Learning-Based Code Understanding

• URL: http://arxiv.org/abs/2402.15769v2
• Date: Mon, 11 Nov 2024 09:38:23 GMT
• Title: GenCode: A Generic Data Augmentation Framework for Boosting Deep Learning-Based Code Understanding
• Authors: Zeming Dong, Qiang Hu, Xiaofei Xie, Maxime Cordy, Mike Papadakis, Jianjun Zhao,
• Abstract summary: We introduce a generic data augmentation framework, GenCode, to enhance the training of code understanding models. To evaluate the effectiveness of GenCode, we conduct experiments on four code understanding tasks and three pre-trained code models. Compared to the state-of-the-art (SOTA) code augmentation method, MixCode, GenCode produces code models with 2.92% higher accuracy and 4.90% robustness on average.
• Score: 28.02426812004216
• License:
• Abstract: Pre-trained code models lead the era of code intelligence with multiple models have been designed with impressive performance. However, one important problem, data augmentation for code data that automatically helps developers prepare training data lacks study in this field. In this paper, we introduce a generic data augmentation framework, GenCode, to enhance the training of code understanding models. Simply speaking, GenCode follows a generation-and-selection paradigm to prepare useful training code data. Specifically, it employs code transformation techniques to generate new code candidates first and then selects important ones as the training data by importance metrics. To evaluate the effectiveness of GenCode, we conduct experiments on four code understanding tasks (e.g., code clone detection) and three pre-trained code models (e.g., CodeT5). Compared to the state-of-the-art (SOTA) code augmentation method, MixCode, GenCode produces code models with 2.92% higher accuracy and 4.90% robustness on average.

Related papers

• Contextualized Data-Wrangling Code Generation in Computational Notebooks [131.26365849822932]
  We propose an automated approach, CoCoMine, to mine data-wrangling code generation examples with clear multi-modal contextual dependency. We construct CoCoNote, a dataset containing 58,221 examples for Contextualized Data-wrangling Code generation in Notebooks. Experiment results demonstrate the significance of incorporating data context in data-wrangling code generation.
  arXiv  Detail & Related papers  (2024-09-20T14:49:51Z)
• Does Your Neural Code Completion Model Use My Code? A Membership Inference Approach [66.51005288743153]
  We investigate the legal and ethical issues of current neural code completion models. We tailor a membership inference approach (termed CodeMI) that was originally crafted for classification tasks. We evaluate the effectiveness of this adapted approach across a diverse array of neural code completion models.
  arXiv  Detail & Related papers  (2024-04-22T15:54:53Z)
• CCT5: A Code-Change-Oriented Pre-Trained Model [14.225942520238936]
  We propose to pre-train a model specially designed for code changes to better support developers in software maintenance. We first collect a large-scale dataset containing 1.5M+ pairwise data of code changes and commit messages. We fine-tune the pre-trained model, CCT5, on three widely-labelled tasks incurred by code changes and two tasks specific to the code review process.
  arXiv  Detail & Related papers  (2023-05-18T07:55:37Z)
• Code Execution with Pre-trained Language Models [88.04688617516827]
  Most pre-trained models for code intelligence ignore the execution trace and only rely on source code and syntactic structures. We develop a mutation-based data augmentation technique to create a large-scale and realistic Python dataset and task for code execution. We then present CodeExecutor, a Transformer model that leverages code execution pre-training and curriculum learning to enhance its semantic comprehension.
  arXiv  Detail & Related papers  (2023-05-08T10:00:05Z)
• CodeExp: Explanatory Code Document Generation [94.43677536210465]
  Existing code-to-text generation models produce only high-level summaries of code. We conduct a human study to identify the criteria for high-quality explanatory docstring for code. We present a multi-stage fine-tuning strategy and baseline models for the task.
  arXiv  Detail & Related papers  (2022-11-25T18:05:44Z)
• Soft-Labeled Contrastive Pre-training for Function-level Code Representation [127.71430696347174]
  We present textbfSCodeR, a textbfSoft-labeled contrastive pre-training framework with two positive sample construction methods. Considering the relevance between codes in a large-scale code corpus, the soft-labeled contrastive pre-training can obtain fine-grained soft-labels. SCodeR achieves new state-of-the-art performance on four code-related tasks over seven datasets.
  arXiv  Detail & Related papers  (2022-10-18T05:17:37Z)
• Enhancing Code Classification by Mixup-Based Data Augmentation [16.49710700412084]
  We propose a Mixup-based data augmentation approach, MixCode, to enhance the source code classification task. We evaluate MixCode on two programming languages (JAVA and Python), two code tasks (problem classification and bug detection), four datasets (JAVA250, Python800, CodRep1, and Refactory) and five model architectures.
  arXiv  Detail & Related papers  (2022-10-06T15:47:54Z)
• NatGen: Generative pre-training by "Naturalizing" source code [18.410818213965918]
  We propose a new pre-training objective, "Naturalizing" of source code. Unlike natural language, code's bimodal, dual-channel nature allows us to generate semantically equivalent code at scale. We fine-tune our model in three generative Software Engineering tasks to achieve state-of-the-art performance rivaling CodeT5.
  arXiv  Detail & Related papers  (2022-06-15T15:08:29Z)
• Enhancing Semantic Code Search with Multimodal Contrastive Learning and Soft Data Augmentation [50.14232079160476]
  We propose a new approach with multimodal contrastive learning and soft data augmentation for code search. We conduct extensive experiments to evaluate the effectiveness of our approach on a large-scale dataset with six programming languages.
  arXiv  Detail & Related papers  (2022-04-07T08:49:27Z)
• InferCode: Self-Supervised Learning of Code Representations by Predicting Subtrees [17.461451218469062]
  This paper proposes InferCode to overcome the limitation by adapting the self-language learning mechanism to build source code model. Subtrees in ASTs are treated with InferCode as the labels for training code representations without any human labeling effort or the overhead of expensive graph construction. Compared to previous code learning techniques applied to the same downstream tasks, such as Code2Vec, Code2Seq, ASTNN, higher performance results are achieved using our pre-trained InferCode model.
  arXiv  Detail & Related papers  (2020-12-13T10:33:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.