The Matthew Effect of AI Programming Assistants: A Hidden Bias in Software Evolution

• URL: http://arxiv.org/abs/2509.23261v2
• Date: Mon, 13 Oct 2025 13:51:00 GMT
• Title: The Matthew Effect of AI Programming Assistants: A Hidden Bias in Software Evolution
• Authors: Fei Gu, Zi Liang, Hongzong LI, Jiahao MA,
• Abstract summary: We conduct large-scale experiments on thousands of algorithmic programming tasks and hundreds of framework selection tasks to investigate how AI-assisted programming interacts with the software ecosystem.<n>Our analysis reveals textbfa striking Matthew effect: the more popular a programming language or framework, the higher the success rate of LLM-generated code.<n>The phenomenon suggests that AI systems may reinforce existing popularity hierarchies, accelerating convergence around dominant tools while hindering diversity and innovation.
• Score: 7.753573982185398
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: AI-assisted programming is rapidly reshaping software development, with large language models (LLMs) enabling new paradigms such as vibe coding and agentic coding. While prior works have focused on prompt design and code generation quality, the broader impact of LLM-driven development on the iterative dynamics of software engineering remains underexplored. In this paper, we conduct large-scale experiments on thousands of algorithmic programming tasks and hundreds of framework selection tasks to systematically investigate how AI-assisted programming interacts with the software ecosystem. Our analysis reveals \textbf{a striking Matthew effect: the more popular a programming language or framework, the higher the success rate of LLM-generated code}. The phenomenon suggests that AI systems may reinforce existing popularity hierarchies, accelerating convergence around dominant tools while hindering diversity and innovation. We provide a quantitative characterization of this effect and discuss its implications for the future evolution of programming ecosystems.

Related papers

• A Survey of Vibe Coding with Large Language Models [93.88284590533242]
  "Vibe Coding" is a development methodology where developers validate AI-generated implementations through outcome observation.<n>Despite its transformative potential, the effectiveness of this emergent paradigm remains under-explored.<n>This survey provides the first comprehensive and systematic review of Vibe Coding with large language models.
  arXiv  Detail & Related papers  (2025-10-14T11:26:56Z)
• Generative AI and the Transformation of Software Development Practices [0.0]
  Generative AI is reshaping how software is designed, written, and maintained.<n>This paper examines how AI-assisted techniques are changing software engineering practice.
  arXiv  Detail & Related papers  (2025-10-12T22:02:10Z)
• Barbarians at the Gate: How AI is Upending Systems Research [58.95406995634148]
  We argue that systems research, long focused on designing and evaluating new performance-oriented algorithms, is particularly well-suited for AI-driven solution discovery.<n>We term this approach as AI-Driven Research for Systems ( ADRS), which iteratively generates, evaluates, and refines solutions.<n>Our results highlight both the disruptive potential and the urgent need to adapt systems research practices in the age of AI.
  arXiv  Detail & Related papers  (2025-10-07T17:49:24Z)
• Rethinking Technology Stack Selection with AI Coding Proficiency [49.617080246389605]
  Large language models (LLMs) are now an integral part of software development.<n>We propose the concept, AI coding proficiency, the degree to which LLMs can utilize a given technology to generate high-quality code snippets.<n>We conduct the first comprehensive empirical study examining AI proficiency across 170 third-party libraries and 61 task scenarios.
  arXiv  Detail & Related papers  (2025-09-14T06:56:47Z)
• A Survey on Code Generation with LLM-based Agents [61.474191493322415]
  Code generation agents powered by large language models (LLMs) are revolutionizing the software development paradigm.<n>LLMs are characterized by three core features.<n>This paper presents a systematic survey of the field of LLM-based code generation agents.
  arXiv  Detail & Related papers  (2025-07-31T18:17:36Z)
• Vibe Coding as a Reconfiguration of Intent Mediation in Software Development: Definition, Implications, and Research Agenda [4.451779041553598]
  vibe coding is a software development paradigm where humans and generative AI engage in collaborative flow to co-create software artifacts.<n>We show that vibe coding reconfigures cognitive work by redistributing labor between humans and machines.<n>We identify key opportunities, including democratization, acceleration, and systemic leverage, alongside risks.
  arXiv  Detail & Related papers  (2025-07-29T15:44:55Z)
• Vibe Coding vs. Agentic Coding: Fundamentals and Practical Implications of Agentic AI [0.36868085124383626]
  Review presents a comprehensive analysis of two emerging paradigms in AI-assisted software development: vibe coding and agentic coding.<n> Vibe coding emphasizes intuitive, human-in-the-loop interaction through prompt-based, conversational interaction.<n>Agentic coding enables autonomous software development through goal-driven agents capable of planning, executing, testing, and iterating tasks with minimal human intervention.
  arXiv  Detail & Related papers  (2025-05-26T03:00:21Z)
• A Path Less Traveled: Reimagining Software Engineering Automation via a Neurosymbolic Paradigm [9.900581015679935]
  We propose Neurosymbolic Software Engineering as a promising paradigm combining neural learning with symbolic (rule-based) reasoning.<n>This hybrid methodology aims to enhance efficiency, reliability, and transparency in AI-driven software engineering.
  arXiv  Detail & Related papers  (2025-05-04T22:10:21Z)
• Generative AI and Empirical Software Engineering: A Paradigm Shift [8.65285948382426]
  The widespread adoption of generative AI in software engineering marks a paradigm shift.<n>This paper examines how integrating AI into software engineering challenges traditional research paradigms.
  arXiv  Detail & Related papers  (2025-02-12T04:13:07Z)
• Agent-Driven Automatic Software Improvement [55.2480439325792]
  This research proposal aims to explore innovative solutions by focusing on the deployment of agents powered by Large Language Models (LLMs) The iterative nature of agents, which allows for continuous learning and adaptation, can help surpass common challenges in code generation. We aim to use the iterative feedback in these systems to further fine-tune the LLMs underlying the agents, becoming better aligned to the task of automated software improvement.
  arXiv  Detail & Related papers  (2024-06-24T15:45:22Z)
• Making Software Development More Diverse and Inclusive: Key Themes, Challenges, and Future Directions [50.545824691484796]
  We identify six themes around the theme challenges and opportunities to improve Software Developer Diversity and Inclusion (SDDI)<n>We identify benefits, harms, and future research directions for the four main themes.<n>We discuss the remaining two themes, Artificial Intelligence & SDDI and AI & Computer Science education, which have a cross-cutting effect on the other themes.
  arXiv  Detail & Related papers  (2024-04-10T16:18:11Z)
• Exploring the intersection of Generative AI and Software Development [0.0]
  The synergy between generative AI and Software Engineering emerges as a transformative frontier. This whitepaper delves into the unexplored realm, elucidating how generative AI techniques can revolutionize software development. It serves as a guide for stakeholders, urging discussions and experiments in the application of generative AI in Software Engineering.
  arXiv  Detail & Related papers  (2023-12-21T19:23:23Z)
• Machine Psychology [54.287802134327485]
  We argue that a fruitful direction for research is engaging large language models in behavioral experiments inspired by psychology. We highlight theoretical perspectives, experimental paradigms, and computational analysis techniques that this approach brings to the table. It paves the way for a "machine psychology" for generative artificial intelligence (AI) that goes beyond performance benchmarks.
  arXiv  Detail & Related papers  (2023-03-24T13:24:41Z)
• Technology Readiness Levels for Machine Learning Systems [107.56979560568232]
  Development and deployment of machine learning systems can be executed easily with modern tools, but the process is typically rushed and means-to-an-end. We have developed a proven systems engineering approach for machine learning development and deployment. Our "Machine Learning Technology Readiness Levels" framework defines a principled process to ensure robust, reliable, and responsible systems.
  arXiv  Detail & Related papers  (2021-01-11T15:54:48Z)
• Distributed and Democratized Learning: Philosophy and Research Challenges [80.39805582015133]
  We propose a novel design philosophy called democratized learning (Dem-AI) Inspired by the societal groups of humans, the specialized groups of learning agents in the proposed Dem-AI system are self-organized in a hierarchical structure to collectively perform learning tasks more efficiently. We present a reference design as a guideline to realize future Dem-AI systems, inspired by various interdisciplinary fields.
  arXiv  Detail & Related papers  (2020-03-18T08:45:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.