GitHub Copilot and Developer Productivity: An Observational Dose-Response Analysis

• URL: http://arxiv.org/abs/2606.00438v1
• Date: Sat, 30 May 2026 00:09:51 GMT
• Title: GitHub Copilot and Developer Productivity: An Observational Dose-Response Analysis
• Abstract summary: GitHub Copilot (GHCP) makes engineers more productive, or do the engineers who use it more differ from those who use it less?<n>We study these questions using 43 weeks of data from 16,223 software engineers across Microsoft's Cloud+AI organization.
• Score: 0.5437663254559298
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Does GitHub Copilot (GHCP) make engineers more productive, or do the engineers who use it more differ from those who use it less? And even within a single engineer, are GHCP-heavy weeks just busy weeks in which more of everything gets done? We study these questions using 43 weeks of data from 16,223 software engineers across Microsoft's Cloud+AI organization. Engineer fixed effects address the first concern by comparing each engineer against themselves rather than against other engineers, eliminating time-invariant differences in skill, role, and team. Active coding time and browser time then enter a Poisson Pseudo-Maximum Likelihood model with two-way fixed effects to address the harder, within-engineer confound: that GHCP-heavy weeks coincide with high-effort weeks. This defines our estimand as an efficiency effect: more pull requests completed at equivalent levels of coding time. Engineers are estimated to complete 40.5% more PRs in their highest GHCP usage weeks relative to their zero-usage weeks, holding measured development effort constant. The gradient is monotonic with diminishing returns at high intensity. Seven robustness and falsification tests target the remaining plausible alternative explanations (non-coding AI engagement, team-level shocks, within-week task reallocation, cross-week contamination, PR slicing into smaller units, shifts toward easier task types, and sensitivity to how the treatment is operationalized). Under an explicitly stated conditional-independence assumption, the within-engineer design estimates a tool-specific efficiency effect that is consistent with all seven robustness tests.

Related papers

• ETA: A New Agentic Paradigm for Embodied Tasks [76.25893120982523]
  We introduce the Embodied Task Agent (ETA), a new paradigm for extending digital agents into the physical world.<n>ETA centers the robot around a Planner that chooses one Tool call at a time, an Interface that controls execution, and a World that returns the result and a fresh observation.<n>ETA provides replaceable Planners, composable Tools and Skills, auditable memory, replayable trajectories, and common interfaces for simulation and real robots.
  arXiv  Detail & Related papers  (2026-08-04T16:56:09Z)
• MirrorCode: AI can rebuild entire programs from behavior alone [1.052679131856939]
  MirrorCode is a long-horizon coding benchmark based on reimplementing entire software projects.<n>Existing AI models can already reimplement complex software, with the strongest model scoring 56% across the benchmark.<n>We show that AI agents can already complete long-horizon software engineering tasks, especially when requirements are precisely specified.
  arXiv  Detail & Related papers  (2026-06-29T11:57:32Z)
• LongCLI-Bench: A Preliminary Benchmark and Study for Long-horizon Agentic Programming in Command-Line Interfaces [65.11019654023978]
  LongCLI-Bench is a benchmark designed to evaluate agentic capabilities across long-horizon, realistic tasks.<n>We curated 20 high-quality, long-horizon tasks from over 1,000 computer science assignments and real-world tasks.<n>Experiments reveal that even state-of-the-art agents achieve pass rates below 20% in LongCLI-Bench.
  arXiv  Detail & Related papers  (2026-02-15T23:12:57Z)
• AIRS-Bench: a Suite of Tasks for Frontier AI Research Science Agents [49.67355440164857]
  We introduce AIRS-Bench, a suite of 20 tasks sourced from state-of-the-art machine learning papers.<n>Airs-Bench tasks assess agentic capabilities over the full research lifecycle.<n>We open-source the AIRS-Bench task definitions and evaluation code to catalyze further development in autonomous scientific research.
  arXiv  Detail & Related papers  (2026-02-06T16:45:02Z)
• Evolving with AI: A Longitudinal Analysis of Developer Logs [3.7353323067733473]
  We study how sustained AI use reshapes actual daily coding practices in the long term.<n>We analyze five dimensions of workflow change: productivity, code quality, code editing, code reuse, and context switching.<n>Our results offer empirical insights into the silent restructuring of software and provide implications for designing future AI-augmented tooling.
  arXiv  Detail & Related papers  (2026-01-15T10:30:24Z)
• Does AI-Assisted Coding Deliver? A Difference-in-Differences Study of Cursor's Impact on Software Projects [18.271710138834536]
  We find that the adoption of Cursor leads to a significant, large, but transient increase in project-level development velocity.<n>The increase in static analysis warnings and code complexity acts as a major factor causing long-term velocity slowdown.
  arXiv  Detail & Related papers  (2025-11-06T15:00:51Z)
• Reinforcement Learning for Machine Learning Engineering Agents [52.03168614623642]
  We show that agents backed by weaker models that improve via reinforcement learning can outperform agents backed by much larger, but static models.<n>We propose duration- aware gradient updates in a distributed asynchronous RL framework to amplify high-cost but high-reward actions.<n>We also propose environment instrumentation to offer partial credit, distinguishing almost-correct programs from those that fail early.
  arXiv  Detail & Related papers  (2025-09-01T18:04:10Z)
• Impact of the Availability of ChatGPT on Software Development: A Synthetic Difference in Differences Estimation using GitHub Data [49.1574468325115]
  ChatGPT is an AI tool that enhances software production efficiency. We estimate ChatGPT's effects on the number of git pushes, repositories, and unique developers per 100,000 people. These results suggest that AI tools like ChatGPT can substantially boost developer productivity, though further analysis is needed to address potential downsides such as low quality code and privacy concerns.
  arXiv  Detail & Related papers  (2024-06-16T19:11:15Z)
• Impact of AI-tooling on the Engineering Workspace [0.0]
  Significant changes were observed in coding time fractions among Copilot users. Some companies experienced a decrease in PR pickup times by up to 33%. One company experienced a shift of up to 17% of effort from maintenance and support work towards product growth initiatives.
  arXiv  Detail & Related papers  (2024-06-11T20:04:09Z)
• The Impact of AI Tool on Engineering at ANZ Bank An Empirical Study on GitHub Copilot within Corporate Environment [0.0]
  This study explores the integration of AI tools in software engineering practices within a large organization. We focus on ANZ Bank, which employs over 5000 engineers covering all aspects of the software development life cycle. This paper details an experiment conducted using GitHub Copilot, a notable AI tool, within a controlled environment to evaluate its effectiveness in real-world engineering tasks.
  arXiv  Detail & Related papers  (2024-02-08T12:47:57Z)
• Straggler-aware Distributed Learning: Communication Computation Latency Trade-off [56.08535873173518]
  Straggling workers can be tolerated by assigning redundant computations and coding across data and computations. In most existing schemes, each non-straggling worker transmits one message per iteration to the parameter server (PS) after completing all its computations. Imposing such a limitation results in two main drawbacks; over-computation due to inaccurate prediction of the straggling behaviour, and under-utilization due to treating workers as straggler/non-straggler.
  arXiv  Detail & Related papers  (2020-04-10T08:39:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.