What Makes a Code Review Useful to OpenDev Developers? An Empirical
Investigation
- URL: http://arxiv.org/abs/2302.11686v2
- Date: Mon, 19 Jun 2023 19:53:15 GMT
- Title: What Makes a Code Review Useful to OpenDev Developers? An Empirical
Investigation
- Authors: Asif Kamal Turzo and Amiangshu Bosu
- Abstract summary: Even a minor improvement in the effectiveness of Code Reviews can incur significant savings for a software development organization.
This study aims to develop a finer grain understanding of what makes a code review comment useful to OSS developers.
- Score: 4.061135251278187
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Context: Due to the association of significant efforts, even a minor
improvement in the effectiveness of Code Reviews(CR) can incur significant
savings for a software development organization. Aim: This study aims to
develop a finer grain understanding of what makes a code review comment useful
to OSS developers, to what extent a code review comment is considered useful to
them, and how various contextual and participant-related factors influence its
usefulness level. Method: On this goal, we have conducted a three-stage
mixed-method study. We randomly selected 2,500 CR comments from the OpenDev
Nova project and manually categorized the comments. We designed a survey of
OpenDev developers to better understand their perspectives on useful CRs.
Combining our survey-obtained scores with our manually labeled dataset, we
trained two regression models - one to identify factors that influence the
usefulness of CR comments and the other to identify factors that improve the
odds of `Functional' defect identification over the others. Key findings: The
results of our study suggest that a CR comment's usefulness is dictated not
only by its technical contributions such as defect findings or quality
improvement tips but also by its linguistic characteristics such as
comprehensibility and politeness. While a reviewer's coding experience
positively associates with CR usefulness, the number of mutual reviews, comment
volume in a file, the total number of lines added /modified, and CR interval
has the opposite associations. While authorship and reviewership experiences
for the files under review have been the most popular attributes for reviewer
recommendation systems, we do not find any significant association of those
attributes with CR usefulness.
Related papers
- Rethinking the Evaluation of Dialogue Systems: Effects of User Feedback on Crowdworkers and LLMs [57.16442740983528]
In ad-hoc retrieval, evaluation relies heavily on user actions, including implicit feedback.
The role of user feedback in annotators' assessment of turns in a conversational perception has been little studied.
We focus on how the evaluation of task-oriented dialogue systems ( TDSs) is affected by considering user feedback, explicit or implicit, as provided through the follow-up utterance of a turn being evaluated.
arXiv Detail & Related papers (2024-04-19T16:45:50Z) - Team-related Features in Code Review Prediction Models [10.576931077314887]
We evaluate the prediction power of features related to code ownership, workload, and team relationship.
Our results show that, individually, features related to code ownership have the best prediction power.
We conclude that all proposed features together with lines of code can make the best predictions for both reviewer participation and amount of feedback.
arXiv Detail & Related papers (2023-12-11T09:30:09Z) - Demystifying Code Snippets in Code Reviews: A Study of the OpenStack and Qt Communities and A Practitioner Survey [6.091233191627442]
We conduct a mixed-methods study to mine information and knowledge related to code snippets in code reviews.
The study results highlight that reviewers can provide code snippets in appropriate scenarios to meet developers' specific information needs in code reviews.
arXiv Detail & Related papers (2023-07-26T17:49:19Z) - Towards Automated Classification of Code Review Feedback to Support
Analytics [4.423428708304586]
This study aims to develop an automated code review comment classification system.
We trained and evaluated supervised learning-based DNN models leveraging code context, comment text, and a set of code metrics.
Our approach outperforms Fregnan et al.'s approach by achieving 18.7% higher accuracy.
arXiv Detail & Related papers (2023-07-07T21:53:20Z) - Exploring the Advances in Identifying Useful Code Review Comments [0.0]
This paper reflects the evolution of research on the usefulness of code review comments.
It examines papers that define the usefulness of code review comments, mine and annotate datasets, study developers' perceptions, analyze factors from different aspects, and use machine learning classifiers to automatically predict the usefulness of code review comments.
arXiv Detail & Related papers (2023-07-03T00:41:20Z) - Towards Personalized Review Summarization by Modeling Historical Reviews
from Customer and Product Separately [59.61932899841944]
Review summarization is a non-trivial task that aims to summarize the main idea of the product review in the E-commerce website.
We propose the Heterogeneous Historical Review aware Review Summarization Model (HHRRS)
We employ a multi-task framework that conducts the review sentiment classification and summarization jointly.
arXiv Detail & Related papers (2023-01-27T12:32:55Z) - Predicting Code Review Completion Time in Modern Code Review [12.696276129130332]
Modern Code Review (MCR) is being adopted in both open source and commercial projects as a common practice.
Code reviews can experience significant delays to be completed due to various socio-technical factors.
There is a lack of tool support to help developers estimating the time required to complete a code review.
arXiv Detail & Related papers (2021-09-30T14:00:56Z) - SIFN: A Sentiment-aware Interactive Fusion Network for Review-based Item
Recommendation [48.1799451277808]
We propose a Sentiment-aware Interactive Fusion Network (SIFN) for review-based item recommendation.
We first encode user/item reviews via BERT and propose a light-weighted sentiment learner to extract semantic features of each review.
Then, we propose a sentiment prediction task that guides the sentiment learner to extract sentiment-aware features via explicit sentiment labels.
arXiv Detail & Related papers (2021-08-18T08:04:38Z) - Deep Just-In-Time Inconsistency Detection Between Comments and Source
Code [51.00904399653609]
In this paper, we aim to detect whether a comment becomes inconsistent as a result of changes to the corresponding body of code.
We develop a deep-learning approach that learns to correlate a comment with code changes.
We show the usefulness of our approach by combining it with a comment update model to build a more comprehensive automatic comment maintenance system.
arXiv Detail & Related papers (2020-10-04T16:49:28Z) - A Unified Dual-view Model for Review Summarization and Sentiment
Classification with Inconsistency Loss [51.448615489097236]
Acquiring accurate summarization and sentiment from user reviews is an essential component of modern e-commerce platforms.
We propose a novel dual-view model that jointly improves the performance of these two tasks.
Experiment results on four real-world datasets from different domains demonstrate the effectiveness of our model.
arXiv Detail & Related papers (2020-06-02T13:34:11Z) - Code Review in the Classroom [57.300604527924015]
Young developers in a classroom setting provide a clear picture of the potential favourable and problematic areas of the code review process.
Their feedback suggests that the process has been well received with some points to better the process.
This paper can be used as guidelines to perform code reviews in the classroom.
arXiv Detail & Related papers (2020-04-19T06:07:45Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.