Impact of AI-tooling on the Engineering Workspace

• URL: http://arxiv.org/abs/2406.07683v1
• Date: Tue, 11 Jun 2024 20:04:09 GMT
• Title: Impact of AI-tooling on the Engineering Workspace
• Authors: Lena Chretien, Nikolas Albarran,
• Abstract summary: Significant changes were observed in coding time fractions among Copilot users. Some companies experienced a decrease in PR pickup times by up to 33%. One company experienced a shift of up to 17% of effort from maintenance and support work towards product growth initiatives.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: To understand the impacts of AI-driven coding tools on engineers' workflow and work environment, we utilize the Jellyfish platform to analyze indicators of change. Key indicators are derived from Allocations, Coding Fraction vs. PR Fraction, Lifecycle Phases, Cycle Time, Jira ticket size, PR pickup time, PR comments, PR comment count, interactions, and coding languages. Significant changes were observed in coding time fractions among Copilot users, with an average decrease of 3% with individual decreases as large as 15%. Ticket sizes decreased by an average of 16% across four companies, accompanied by an 8% decrease in cycle times, whereas the control group showed no change. Additionally, the PR process evolved with Copilot usage, featuring longer and more comprehensive comments, despite the weekly number of PRs reviewed remaining constant. Not all hypothesized changes were observed across all participating companies. However, some companies experienced a decrease in PR pickup times by up to 33%, indicating reduced workflow bottlenecks, and one company experienced a shift of up to 17% of effort from maintenance and support work towards product growth initiatives. This study is the first to utilize data from more than one company and goes beyond simple productivity and satisfaction measures, considering real-world engineering settings instead. By doing so, we highlight that some companies seem to benefit more than others from the use of Copilot and that changes can be subtle when investigating aggregates rather than specific aspects of engineering work and workflows - something that will be further investigated in the future.

Related papers

• Analyzing DevOps Practices Through Merge Request Data: A Case Study in Networking Software Company [2.5999037208435705]
  GitLab's Request (MR) mechanism streamlines code submission and review. MR data reflects broader aspects, including collaboration patterns, productivity, and process optimization. This study examines 26.7k MRs from four teams across 116 projects of a networking software company.
  arXiv  Detail & Related papers  (2025-03-18T19:33:34Z)
• How much does AI impact development speed? An enterprise-based randomized controlled trial [8.759453531975668]
  We estimate the impact of three AI features on the time developers spent on a complex, enterprise-grade task. We also found an interesting effect whereby developers who spend more hours on code-related activities per day were faster with AI.
  arXiv  Detail & Related papers  (2024-10-16T18:31:14Z)
• GTPT: Group-based Token Pruning Transformer for Efficient Human Pose Estimation [46.74217876359835]
  Group-based Token Pruning Transformer (GTPT) for efficient human pose estimation. Group-based Token Pruning Transformer (GTPT) for efficient human pose estimation.
  arXiv  Detail & Related papers  (2024-07-15T14:32:45Z)
• Transforming Software Development: Evaluating the Efficiency and Challenges of GitHub Copilot in Real-World Projects [0.0]
  GitHub Copilot is an AI-powered coding assistant. This study evaluates the efficiency gains, areas for improvement, and emerging challenges of using GitHub Copilot.
  arXiv  Detail & Related papers  (2024-06-25T19:51:21Z)
• Impact of the Availability of ChatGPT on Software Development: A Synthetic Difference in Differences Estimation using GitHub Data [49.1574468325115]
  ChatGPT is an AI tool that enhances software production efficiency. We estimate ChatGPT's effects on the number of git pushes, repositories, and unique developers per 100,000 people. These results suggest that AI tools like ChatGPT can substantially boost developer productivity, though further analysis is needed to address potential downsides such as low quality code and privacy concerns.
  arXiv  Detail & Related papers  (2024-06-16T19:11:15Z)
• Generative AI for Pull Request Descriptions: Adoption, Impact, and Developer Interventions [11.620351603683496]
  GitHub's Copilot for Pull Requests (PRs) is a promising service aiming to automate various developer tasks related to PRs. In this study, we examine 18,256 PRs in which parts of the descriptions were crafted by generative AI. Our findings indicate that Copilot for PRs, though in its infancy, is seeing a marked uptick in adoption.
  arXiv  Detail & Related papers  (2024-02-14T06:20:57Z)
• The Impact of AI Tool on Engineering at ANZ Bank An Empirical Study on GitHub Copilot within Corporate Environment [0.0]
  This study explores the integration of AI tools in software engineering practices within a large organization. We focus on ANZ Bank, which employs over 5000 engineers covering all aspects of the software development life cycle. This paper details an experiment conducted using GitHub Copilot, a notable AI tool, within a controlled environment to evaluate its effectiveness in real-world engineering tasks.
  arXiv  Detail & Related papers  (2024-02-08T12:47:57Z)
• Does Code Review Speed Matter for Practitioners? [0.0]
  Increasing code velocity is a common goal for a variety of software projects. We conducted a survey to study the code velocity-related beliefs and practices in place.
  arXiv  Detail & Related papers  (2023-11-04T19:22:23Z)
• Rethinking People Analytics With Inverse Transparency by Design [57.67333075002697]
  We propose a new design approach for workforce analytics we refer to as inverse transparency by design. We find that architectural changes are made without inhibiting core functionality. We conclude that inverse transparency by design is a promising approach to realize accepted and responsible people analytics.
  arXiv  Detail & Related papers  (2023-05-16T21:37:35Z)
• Best Practices for 2-Body Pose Forecasting [58.661899246497896]
  We review the progress in human pose forecasting and provide an in-depth assessment of the single-person practices that perform best. Other single-person practices do not transfer to 2-body, so the proposed best ones do not include hierarchical body modeling or attention-based interaction encoding. Our proposed 2-body pose forecasting best practices yield a performance improvement of 21.9% over the state-of-the-art on the most recent ExPI dataset.
  arXiv  Detail & Related papers  (2023-04-12T10:46:23Z)
• CLUTR: Curriculum Learning via Unsupervised Task Representation Learning [130.79246770546413]
  CLUTR is a novel curriculum learning algorithm that decouples task representation and curriculum learning into a two-stage optimization. We show CLUTR outperforms PAIRED, a principled and popular UED method, in terms of generalization and sample efficiency in the challenging CarRacing and navigation environments.
  arXiv  Detail & Related papers  (2022-10-19T01:45:29Z)
• Spatio-Temporal Graph Representation Learning for Fraudster Group Detection [50.779498955162644]
  Companies may hire fraudster groups to write fake reviews to either demote competitors or promote their own businesses. To detect such groups, a common model is to represent fraudster groups' static networks. We propose to first capitalize on the effectiveness of the HIN-RNN in both reviewers' representation learning.
  arXiv  Detail & Related papers  (2022-01-07T08:01:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.