Beyond Code: The Multidimensional Impacts of Large Language Models in Software Development

• URL: http://arxiv.org/abs/2506.22704v2
• Date: Tue, 01 Jul 2025 02:35:48 GMT
• Title: Beyond Code: The Multidimensional Impacts of Large Language Models in Software Development
• Authors: Sardar Bonabi, Sarah Bana, Vijay Gurbaxani, Tingting Nian,
• Abstract summary: Large language models (LLMs) are poised to significantly impact software development, especially in the Open-Source Software (OSS) sector.<n>We first outline the mechanisms through which LLMs may influence OSS through code development, collaborative knowledge transfer, and skill development.<n>We then empirically examine how LLMs affect OSS developers' work in these three key areas.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) are poised to significantly impact software development, especially in the Open-Source Software (OSS) sector. To understand this impact, we first outline the mechanisms through which LLMs may influence OSS through code development, collaborative knowledge transfer, and skill development. We then empirically examine how LLMs affect OSS developers' work in these three key areas. Leveraging a natural experiment from a temporary ChatGPT ban in Italy, we employ a Difference-in-Differences framework with two-way fixed effects to analyze data from all OSS developers on GitHub in three similar countries, Italy, France, and Portugal, totaling 88,022 users. We find that access to ChatGPT increases developer productivity by 6.4%, knowledge sharing by 9.6%, and skill acquisition by 8.4%. These benefits vary significantly by user experience level: novice developers primarily experience productivity gains, whereas more experienced developers benefit more from improved knowledge sharing and accelerated skill acquisition. In addition, we find that LLM-assisted learning is highly context-dependent, with the greatest benefits observed in technically complex, fragmented, or rapidly evolving contexts. We show that the productivity effects of LLMs extend beyond direct code generation to include enhanced collaborative learning and knowledge exchange among developers, dynamics that are essential for gaining a holistic understanding of LLMs' impact in OSS. Our findings offer critical managerial implications: strategically deploying LLMs can accelerate novice developers' onboarding and productivity, empower intermediate developers to foster knowledge sharing and collaboration, and support rapid skill acquisition, together enhancing long-term organizational productivity and agility.

Related papers

• Learning to Contextualize Web Pages for Enhanced Decision Making by LLM Agents [89.98593996816186]
  We introduce LCoW, a framework for Learning language models to Contextualize complex Web pages into a more comprehensible form.<n>LCoW decouples web page understanding from decision making by training a separate contextualization module.<n>We demonstrate that our contextualization module effectively integrates with LLM agents of various scales to significantly enhance their decision-making capabilities.
  arXiv  Detail & Related papers  (2025-03-12T01:33:40Z)
• LLMs' Reshaping of People, Processes, Products, and Society in Software Development: A Comprehensive Exploration with Early Adopters [3.4069804433026314]
  Large language models (LLMs) like OpenAI ChatGPT, Google Gemini, and GitHub Copilot are rapidly gaining traction in the software industry.<n>Our study provides a nuanced understanding of how LLMs are shaping the landscape of software development.
  arXiv  Detail & Related papers  (2025-03-06T22:27:05Z)
• The Impact of Generative AI on Collaborative Open-Source Software Development: Evidence from GitHub Copilot [4.8256226973915455]
  Using GitHub's proprietary Copilot usage data, we find that Copilot use increases project-level code contributions by 5.9%.<n>This gain is driven by a 2.1% increase in individual code contributions and a 3.4% rise in developer coding participation.<n>While AI expands who can contribute and how much they contribute, it slows coordination in collective development efforts.
  arXiv  Detail & Related papers  (2024-10-02T23:26:10Z)
• FactorLLM: Factorizing Knowledge via Mixture of Experts for Large Language Models [50.331708897857574]
  We introduce FactorLLM, a novel approach that decomposes well-trained dense FFNs into sparse sub-networks without requiring any further modifications. FactorLLM achieves comparable performance to the source model securing up to 85% model performance while obtaining over a 30% increase in inference speed.
  arXiv  Detail & Related papers  (2024-08-15T16:45:16Z)
• A Study on Developer Behaviors for Validating and Repairing LLM-Generated Code Using Eye Tracking and IDE Actions [13.58143103712]
  GitHub Copilot is a large language model (LLM)-powered code generation tool. This paper investigates how developers validate and repair code generated by Copilot. Being aware of the code's provenance led to improved performance, increased search efforts, more frequent Copilot usage, and higher cognitive workload.
  arXiv  Detail & Related papers  (2024-05-25T06:20:01Z)
• Empowering Large Language Model Agents through Action Learning [85.39581419680755]
  Large Language Model (LLM) Agents have recently garnered increasing interest yet they are limited in their ability to learn from trial and error. We argue that the capacity to learn new actions from experience is fundamental to the advancement of learning in LLM agents. We introduce a framework LearnAct with an iterative learning strategy to create and improve actions in the form of Python functions.
  arXiv  Detail & Related papers  (2024-02-24T13:13:04Z)
• Experiential Co-Learning of Software-Developing Agents [83.34027623428096]
  Large language models (LLMs) have brought significant changes to various domains, especially in software development. We introduce Experiential Co-Learning, a novel LLM-agent learning framework. Experiments demonstrate that the framework enables agents to tackle unseen software-developing tasks more effectively.
  arXiv  Detail & Related papers  (2023-12-28T13:50:42Z)
• Supervised Knowledge Makes Large Language Models Better In-context Learners [94.89301696512776]
  Large Language Models (LLMs) exhibit emerging in-context learning abilities through prompt engineering. The challenge of improving the generalizability and factuality of LLMs in natural language understanding and question answering remains under-explored. We propose a framework that enhances the reliability of LLMs as it: 1) generalizes out-of-distribution data, 2) elucidates how LLMs benefit from discriminative models, and 3) minimizes hallucinations in generative tasks.
  arXiv  Detail & Related papers  (2023-12-26T07:24:46Z)
• MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedback [78.60644407028022]
  We introduce MINT, a benchmark that evaluates large language models' ability to solve tasks with multi-turn interactions. LLMs generally benefit from tools and language feedback, with performance gains of 1-8% for each turn of tool use. LLMs evaluated, supervised instruction-finetuning (SIFT) and reinforcement learning from human feedback (RLHF) generally hurt multi-turn capabilities.
  arXiv  Detail & Related papers  (2023-09-19T15:25:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.