Delving into Commit-Issue Correlation to Enhance Commit Message Generation Models

• URL: http://arxiv.org/abs/2308.00147v2
• Date: Thu, 28 Sep 2023 16:59:23 GMT
• Title: Delving into Commit-Issue Correlation to Enhance Commit Message Generation Models
• Authors: Liran Wang, Xunzhu Tang, Yichen He, Changyu Ren, Shuhua Shi, Chaoran Yan, Zhoujun Li
• Abstract summary: Commit message generation is a challenging task in automated software engineering. tool is a novel paradigm that can introduce the correlation between commits and issues into the training phase of models. The results show that compared with the original models, the performance of tool-enhanced models is significantly improved.
• Score: 13.605167159285374
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Commit message generation (CMG) is a challenging task in automated software engineering that aims to generate natural language descriptions of code changes for commits. Previous methods all start from the modified code snippets, outputting commit messages through template-based, retrieval-based, or learning-based models. While these methods can summarize what is modified from the perspective of code, they struggle to provide reasons for the commit. The correlation between commits and issues that could be a critical factor for generating rational commit messages is still unexplored. In this work, we delve into the correlation between commits and issues from the perspective of dataset and methodology. We construct the first dataset anchored on combining correlated commits and issues. The dataset consists of an unlabeled commit-issue parallel part and a labeled part in which each example is provided with human-annotated rational information in the issue. Furthermore, we propose \tool (\underline{Ex}traction, \underline{Gro}unding, \underline{Fi}ne-tuning), a novel paradigm that can introduce the correlation between commits and issues into the training phase of models. To evaluate whether it is effective, we perform comprehensive experiments with various state-of-the-art CMG models. The results show that compared with the original models, the performance of \tool-enhanced models is significantly improved.

Related papers

• Towards Realistic Evaluation of Commit Message Generation by Matching Online and Offline Settings [77.20838441870151]
  Commit message generation is a crucial task in software engineering that is challenging to evaluate correctly. We use an online metric - the number of edits users introduce before committing the generated messages to the VCS - to select metrics for offline experiments. Our results indicate that edit distance exhibits the highest correlation, whereas commonly used similarity metrics such as BLEU and METEOR demonstrate low correlation.
  arXiv  Detail & Related papers  (2024-10-15T20:32:07Z)
• Robust and Scalable Model Editing for Large Language Models [75.95623066605259]
  We propose EREN (Edit models by REading Notes) to improve the scalability and robustness of LLM editing. Unlike existing techniques, it can integrate knowledge from multiple edits, and correctly respond to syntactically similar but semantically unrelated inputs.
  arXiv  Detail & Related papers  (2024-03-26T06:57:23Z)
• List-aware Reranking-Truncation Joint Model for Search and Retrieval-augmented Generation [80.12531449946655]
  We propose a Reranking-Truncation joint model (GenRT) that can perform the two tasks concurrently. GenRT integrates reranking and truncation via generative paradigm based on encoder-decoder architecture. Our method achieves SOTA performance on both reranking and truncation tasks for web search and retrieval-augmented LLMs.
  arXiv  Detail & Related papers  (2024-02-05T06:52:53Z)
• COMET: Generating Commit Messages using Delta Graph Context Representation [2.5899040911480182]
  Commit messages explain code changes in a commit and facilitate collaboration among developers. We propose Comet, a novel approach that captures context of code changes using a graph-based representation. Tests show Comet outperforms state-of-the-art techniques in terms of bleu-norm and meteor metrics.
  arXiv  Detail & Related papers  (2024-02-02T19:01:52Z)
• Commit Messages in the Age of Large Language Models [0.9217021281095906]
  We evaluate the performance of OpenAI's ChatGPT for generating commit messages based on code changes. We compare the results obtained with ChatGPT to previous automatic commit message generation methods that have been trained specifically on commit data.
  arXiv  Detail & Related papers  (2024-01-31T06:47:12Z)
• Boosting Commit Classification with Contrastive Learning [0.8655526882770742]
  Commit Classification (CC) is an important task in software maintenance. We propose a contrastive learning-based commit classification framework. Our framework can solve the CC problem simply but effectively in fewshot scenarios.
  arXiv  Detail & Related papers  (2023-08-16T10:02:36Z)
• From Commit Message Generation to History-Aware Commit Message Completion [49.175498083165884]
  We argue that if we could shift the focus from commit message generation to commit message completion, we could significantly improve the quality and the personal nature of the resulting commit messages. Since the existing datasets lack historical data, we collect and share a novel dataset called CommitChronicle, containing 10.7M commits across 20 programming languages. Our results show that in some contexts, commit message completion shows better results than generation, and that while in general GPT-3.5-turbo performs worse, it shows potential for long and detailed messages.
  arXiv  Detail & Related papers  (2023-08-15T09:10:49Z)
• DORE: Document Ordered Relation Extraction based on Generative Framework [56.537386636819626]
  This paper investigates the root cause of the underwhelming performance of the existing generative DocRE models. We propose to generate a symbolic and ordered sequence from the relation matrix which is deterministic and easier for model to learn. Experimental results on four datasets show that our proposed method can improve the performance of the generative DocRE models.
  arXiv  Detail & Related papers  (2022-10-28T11:18:10Z)
• Jointly Learning to Repair Code and Generate Commit Message [78.4177637346384]
  We construct a multilingual triple dataset including buggy code, fixed code, and commit messages for this novel task. To deal with the error propagation problem of the cascaded method, the joint model is proposed that can both repair the code and generate the commit message. Experimental results show that the enhanced cascaded model with teacher-student method and multitask-learning method achieves the best score on different metrics of automated code repair.
  arXiv  Detail & Related papers  (2021-09-25T07:08:28Z)
• On the Evaluation of Commit Message Generation Models: An Experimental Study [33.19314967188712]
  Commit messages are natural language descriptions of code changes, which are important for program understanding and maintenance. Various approaches utilizing generation or retrieval techniques have been proposed to automatically generate commit messages. This paper conducts a systematic and in-depth analysis of the state-of-the-art models and datasets.
  arXiv  Detail & Related papers  (2021-07-12T12:38:02Z)
• CoreGen: Contextualized Code Representation Learning for Commit Message Generation [39.383390029545865]
  We propose a novel Contextualized code representation learning strategy for commit message Generation (CoreGen) Experiments on the benchmark dataset demonstrate the superior effectiveness of our model over the baseline models with at least 28.18% improvement in terms of BLEU-4 score.
  arXiv  Detail & Related papers  (2020-07-14T09:43:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.