Meta Learning for Code Summarization

• URL: http://arxiv.org/abs/2201.08310v1
• Date: Thu, 20 Jan 2022 17:23:34 GMT
• Title: Meta Learning for Code Summarization
• Authors: Moiz Rauf, Sebastian Pad\'o, Michael Pradel
• Abstract summary: We show that three SOTA models for code summarization work well on largely disjoint subsets of a large code-base. We propose three meta-models that select the best candidate summary for a given code segment.
• Score: 10.403206672504664
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Source code summarization is the task of generating a high-level natural language description for a segment of programming language code. Current neural models for the task differ in their architecture and the aspects of code they consider. In this paper, we show that three SOTA models for code summarization work well on largely disjoint subsets of a large code-base. This complementarity motivates model combination: We propose three meta-models that select the best candidate summary for a given code segment. The two neural models improve significantly over the performance of the best individual model, obtaining an improvement of 2.1 BLEU points on a dataset of code segments where at least one of the individual models obtains a non-zero BLEU.

Related papers

• Code Representation Learning At Scale [75.04686476303436]
  We fuel code representation learning with a vast amount of code data via a two-stage pretraining scheme. We first train the encoders via a mix that leverages both randomness in masking language modeling and the structure aspect of programming language. We then enhance the representations via contrastive learning with hard negative and hard positive constructed in an unsupervised manner.
  arXiv  Detail & Related papers  (2024-02-02T22:19:15Z)
• LLM-Assisted Code Cleaning For Training Accurate Code Generators [53.087019724256606]
  We investigate data quality for code and find that making the code more structured and readable leads to improved code generation performance of the system. We build a novel data-cleaning pipeline that uses these principles to transform existing programs. We evaluate our approach on two challenging algorithmic code generation benchmarks and find that fine-tuning CodeLLaMa-7B improves the performance by up to 30% compared to fine-tuning on the original dataset.
  arXiv  Detail & Related papers  (2023-11-25T02:45:50Z)
• A Comprehensive Review of State-of-The-Art Methods for Java Code Generation from Natural Language Text [0.0]
  This paper provides a comprehensive review of the evolution and progress of deep learning models in Java code generation task. We focus on the most important methods and present their merits and limitations, as well as the objective functions used by the community.
  arXiv  Detail & Related papers  (2023-06-10T07:27:51Z)
• CodeGen2: Lessons for Training LLMs on Programming and Natural Languages [116.74407069443895]
  We unify encoder and decoder-based models into a single prefix-LM. For learning methods, we explore the claim of a "free lunch" hypothesis. For data distributions, the effect of a mixture distribution and multi-epoch training of programming and natural languages on model performance is explored.
  arXiv  Detail & Related papers  (2023-05-03T17:55:25Z)
• CodeExp: Explanatory Code Document Generation [94.43677536210465]
  Existing code-to-text generation models produce only high-level summaries of code. We conduct a human study to identify the criteria for high-quality explanatory docstring for code. We present a multi-stage fine-tuning strategy and baseline models for the task.
  arXiv  Detail & Related papers  (2022-11-25T18:05:44Z)
• Part-Based Models Improve Adversarial Robustness [57.699029966800644]
  We show that combining human prior knowledge with end-to-end learning can improve the robustness of deep neural networks. Our model combines a part segmentation model with a tiny classifier and is trained end-to-end to simultaneously segment objects into parts. Our experiments indicate that these models also reduce texture bias and yield better robustness against common corruptions and spurious correlations.
  arXiv  Detail & Related papers  (2022-09-15T15:41:47Z)
• Z-Code++: A Pre-trained Language Model Optimized for Abstractive Summarization [108.09419317477986]
  Z-Code++ is a new pre-trained language model optimized for abstractive text summarization. The model is first pre-trained using text corpora for language understanding, and then is continually pre-trained on summarization corpora for grounded text generation. Our model is parameter-efficient in that it outperforms the 600x larger PaLM-540B on XSum, and the finetuned 200x larger GPT3-175B on SAMSum.
  arXiv  Detail & Related papers  (2022-08-21T01:00:54Z)
• Evaluating few shot and Contrastive learning Methods for Code Clone Detection [5.1623866691702744]
  Code Clone Detection is a software engineering task that is used for plagiarism detection, code search, and code comprehension. Deep learning-based models have achieved an F1 score (a metric used to assess classifiers) of $sim$95% on the CodeXGLUE benchmark. No previous study evaluates the generalizability of these models where a limited amount of annotated data is available.
  arXiv  Detail & Related papers  (2022-04-15T15:01:55Z)
• Assemble Foundation Models for Automatic Code Summarization [9.53949558569201]
  We propose a flexible and robust approach for automatic code summarization based on neural networks. We assemble available foundation models, such as CodeBERT and GPT-2, into a single model named AdaMo. We introduce two adaptive schemes from the perspective of knowledge transfer, namely continuous pretraining and intermediate finetuning.
  arXiv  Detail & Related papers  (2022-01-13T21:38:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.