Automated Generation of Commit Messages in Software Repositories

• URL: http://arxiv.org/abs/2504.12998v1
• Date: Thu, 17 Apr 2025 15:08:05 GMT
• Title: Automated Generation of Commit Messages in Software Repositories
• Authors: Varun Kumar Palakodeti, Abbas Heydarnoori,
• Abstract summary: Commit messages are crucial for documenting software changes, aiding in program comprehension and maintenance.<n>Our research presents an automated approach to generate commit messages using Machine Learning (ML) and Natural Language Processing (NLP)<n>We used the dataset of code changes and corresponding commit messages that was used by Liu et al.
• Score: 0.7366405857677226
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Commit messages are crucial for documenting software changes, aiding in program comprehension and maintenance. However, creating effective commit messages is often overlooked by developers due to time constraints and varying levels of documentation skills. Our research presents an automated approach to generate commit messages using Machine Learning (ML) and Natural Language Processing (NLP) by developing models that use techniques such as Logistic Regression with TF-IDF and Word2Vec, as well as more sophisticated methods like LSTM. We used the dataset of code changes and corresponding commit messages that was used by Liu et al., which we used to train and evaluate ML/NLP models and was chosen because it is extensively used in previous research, also for comparability in our study. The objective was to explore which ML/NLP techniques generate the most effective, clear, and concise commit messages that accurately reflect the code changes. We split the dataset into training, validation, and testing sets and used these sets to evaluate the performance of each model using qualitative and quantitative evaluation methods. Our results reveal a spectrum of effectiveness among these models, with the highest BLEU score achieved being 16.82, showcasing the models' capability in automating a clear and concise commit message generation. Our paper offers insights into the comparative effectiveness of different machine learning models for automating commit message generation in software development, aiming to enhance the overall practice of code documentation. The source code is available at https://doi.org/10.5281/zenodo.10888106.

Related papers

• GALOT: Generative Active Learning via Optimizable Zero-shot Text-to-image Generation [21.30138131496276]
  This paper integrates the zero-shot text-to-image (T2I) synthesis and active learning. We leverage the AL criteria to optimize the text inputs for generating more informative and diverse data samples. This approach reduces the cost of data collection and annotation while increasing the efficiency of model training.
  arXiv  Detail & Related papers  (2024-12-18T18:40:21Z)
• CELA: Cost-Efficient Language Model Alignment for CTR Prediction [70.65910069412944]
  Click-Through Rate (CTR) prediction holds a paramount position in recommender systems. Recent efforts have sought to mitigate these challenges by integrating Pre-trained Language Models (PLMs) We propose textbfCost-textbfEfficient textbfLanguage Model textbfAlignment (textbfCELA) for CTR prediction.
  arXiv  Detail & Related papers  (2024-05-17T07:43:25Z)
• Improving Sentence Embeddings with Automatic Generation of Training Data Using Few-shot Examples [13.946626388239443]
  We aim to improve sentence embeddings without using large manually annotated datasets. We will focus on automatic dataset generation through few-shot learning and explore the appropriate methods to leverage few-shot examples.
  arXiv  Detail & Related papers  (2024-02-23T06:33:51Z)
• Towards Automatic Translation of Machine Learning Visual Insights to Analytical Assertions [23.535630175567146]
  We present our vision for developing an automated tool capable of translating visual properties observed in Machine Learning (ML) visualisations into Python assertions. The tool aims to streamline the process of manually verifying these visualisations in the ML development cycle, which is critical as real-world data and assumptions often change post-deployment.
  arXiv  Detail & Related papers  (2024-01-15T14:11:59Z)
• Using Large Language Models for Commit Message Generation: A Preliminary Study [5.5784148764236114]
  Large language models (LLMs) can be used to generate commit messages automatically and effectively. In 78% of the 366 samples, the commit messages generated by LLMs were evaluated by humans as the best.
  arXiv  Detail & Related papers  (2024-01-11T14:06:39Z)
• CLOMO: Counterfactual Logical Modification with Large Language Models [109.60793869938534]
  We introduce a novel task, Counterfactual Logical Modification (CLOMO), and a high-quality human-annotated benchmark. In this task, LLMs must adeptly alter a given argumentative text to uphold a predetermined logical relationship. We propose an innovative evaluation metric, the Self-Evaluation Score (SES), to directly evaluate the natural language output of LLMs.
  arXiv  Detail & Related papers  (2023-11-29T08:29:54Z)
• From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuning [52.257422715393574]
  We introduce a self-guided methodology for Large Language Models (LLMs) to autonomously discern and select cherry samples from open-source datasets. Our key innovation, the Instruction-Following Difficulty (IFD) metric, emerges as a pivotal metric to identify discrepancies between a model's expected responses and its intrinsic generation capability.
  arXiv  Detail & Related papers  (2023-08-23T09:45:29Z)
• The Devil is in the Errors: Leveraging Large Language Models for Fine-grained Machine Translation Evaluation [93.01964988474755]
  AutoMQM is a prompting technique which asks large language models to identify and categorize errors in translations. We study the impact of labeled data through in-context learning and finetuning. We then evaluate AutoMQM with PaLM-2 models, and we find that it improves performance compared to just prompting for scores.
  arXiv  Detail & Related papers  (2023-08-14T17:17:21Z)
• Language models are weak learners [71.33837923104808]
  We show that prompt-based large language models can operate effectively as weak learners. We incorporate these models into a boosting approach, which can leverage the knowledge within the model to outperform traditional tree-based boosting. Results illustrate the potential for prompt-based LLMs to function not just as few-shot learners themselves, but as components of larger machine learning pipelines.
  arXiv  Detail & Related papers  (2023-06-25T02:39:19Z)
• Lexically Aware Semi-Supervised Learning for OCR Post-Correction [90.54336622024299]
  Much of the existing linguistic data in many languages of the world is locked away in non-digitized books and documents. Previous work has demonstrated the utility of neural post-correction methods on recognition of less-well-resourced languages. We present a semi-supervised learning method that makes it possible to utilize raw images to improve performance.
  arXiv  Detail & Related papers  (2021-11-04T04:39:02Z)
• Efficient Nearest Neighbor Language Models [114.40866461741795]
  Non-parametric neural language models (NLMs) learn predictive distributions of text utilizing an external datastore. We show how to achieve up to a 6x speed-up in inference speed while retaining comparable performance.
  arXiv  Detail & Related papers  (2021-09-09T12:32:28Z)
• On the Evaluation of Commit Message Generation Models: An Experimental Study [33.19314967188712]
  Commit messages are natural language descriptions of code changes, which are important for program understanding and maintenance. Various approaches utilizing generation or retrieval techniques have been proposed to automatically generate commit messages. This paper conducts a systematic and in-depth analysis of the state-of-the-art models and datasets.
  arXiv  Detail & Related papers  (2021-07-12T12:38:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.