Debt Behind the AI Boom: A Large-Scale Empirical Study of AI-Generated Code in the Wild

• URL: http://arxiv.org/abs/2603.28592v1
• Date: Mon, 30 Mar 2026 15:38:05 GMT
• Title: Debt Behind the AI Boom: A Large-Scale Empirical Study of AI-Generated Code in the Wild
• Abstract summary: We build a dataset of 304,362 verified AI-authored commits from 6,275 GitHub repositories.<n>We identify 484,606 distinct issues, and that code smells are by far the most common type, accounting for 89.1% of all issues.<n>These findings show that AI-generated code can introduce long-term maintenance costs into real software projects.
• Score: 13.35459155979355
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: AI coding assistants are now widely used in software development. Software developers increasingly integrate AI-generated code into their codebases to improve productivity. Prior studies have shown that AI-generated code may contain code quality issues under controlled settings. However, we still know little about the real-world impact of AI-generated code on software quality and maintenance after it is introduced into production repositories. In other words, it remains unclear whether such issues are quickly fixed or persist and accumulate over time as technical debt. In this paper, we conduct a large-scale empirical study on the technical debt introduced by AI coding assistants in the wild. To achieve that, we built a dataset of 304,362 verified AI-authored commits from 6,275 GitHub repositories, covering five widely used AI coding assistants. For each commit, we run static analysis before and after the change to precisely attribute which code smells, bugs, and security issues the AI introduced. We then track each introduced issue from the introducing commit to the latest repository revision to study its lifecycle. Our results show that we identified 484,606 distinct issues, and that code smells are by far the most common type, accounting for 89.1% of all issues. We also find that more than 15% of commits from every AI coding assistant introduce at least one issue, although the rates vary across tools. More importantly, 24.2% of tracked AI-introduced issues still survive at the latest revision of the repository. These findings show that AI-generated code can introduce long-term maintenance costs into real software projects and highlight the need for stronger quality assurance in AI-assisted development.

Related papers

• Empirical Study on the Characteristics and Evolution of AI-usage in GitHub Repositories: Evidence from Code Comments [7.213400161126317]
  We analyze 35,361 GitHub code comments that explicitly reference AI use and their associated code blocks.<n>We first open-code 500 unique comments and code blocks to derive a taxonomy of AI-assisted development activities.<n>We then annotate the full dataset using two LLM-based classifiers and aggregate predictions with Dawid-Skene expectation-maximization.
  arXiv  Detail & Related papers  (2026-06-05T02:37:44Z)
• AI for Auto-Research: Roadmap & User Guide [107.0834449839233]
  We present an end-to-end analysis of AI across the complete research lifecycle.<n>We identify a sharp, stage-dependent boundary between reliable assistance and unreliable autonomy.<n>We show that greater automation can obscure rather than eliminate failure modes.
  arXiv  Detail & Related papers  (2026-05-18T17:08:26Z)
• To What Extent Does Agent-generated Code Require Maintenance? An Empirical Study [0.0]
  This study empirically investigates the maintenance extent, human involvement, and modification types of AI-generated files versus human-authored code.<n>We analyzed over 1,000 files and approximately 3,200 changes from 100 popular repositories.
  arXiv  Detail & Related papers  (2026-05-07T15:52:41Z)
• SWE-chat: Coding Agent Interactions From Real Users in the Wild [70.18158706281724]
  SWE-chat is the first large-scale dataset of real coding agent sessions collected from open-source developers in the wild.<n>The dataset currently contains 6,000 sessions, comprising more than 63,000 user prompts and 355,000 agent tool calls.
  arXiv  Detail & Related papers  (2026-04-22T17:08:19Z)
• AI builds, We Analyze: An Empirical Study of AI-Generated Build Code Quality [0.0]
  The rapid adoption of AI coding agents for software development has raised important questions about the quality and maintainability of the code they produce.<n>This data mining challenge focuses on AIDev, the first large-scale, openly available dataset capturing agent-pull requests from real-world GitHub repositories.<n>We identified 364 maintainability and security-related build smells across varying severity levels, indicating that AI-generated build code can introduce quality issues.
  arXiv  Detail & Related papers  (2026-01-23T15:40:28Z)
• AI Code in the Wild: Measuring Security Risks and Ecosystem Shifts of AI-Generated Code in Modern Software [12.708926174194199]
  We present the first large-scale empirical study of AI-generated code (AIGCode) in the wild.<n>We build a high-precision detection pipeline and a benchmark to distinguish AIGCode from human-written code.<n>This lets us label commits, files, and functions along a human/AI axis and trace how AIGCode moves through projects and vulnerability life cycles.
  arXiv  Detail & Related papers  (2025-12-21T02:26:29Z)
• Vibe Coding in Practice: Motivations, Challenges, and a Future Outlook -- a Grey Literature Review [2.5195922470930614]
  Vibe coding is the practice where users rely on AI code generation tools through intuition and trial-and-error without necessarily understanding the underlying code.<n>No research has systematically investigated why users engage in vibe coding, what they experience while doing so, and how they approach quality assurance (QA) and perceive the quality of the AI-generated code.<n>Our analysis reveals a speed-quality trade-off paradox, where vibe coders are motivated by speed and accessibility, often experiencing rapid instant success and flow'', yet most perceive the resulting code as fast but flawed.
  arXiv  Detail & Related papers  (2025-09-30T22:35:00Z)
• Code with Me or for Me? How Increasing AI Automation Transforms Developer Workflows [60.04362496037186]
  We present the first controlled study of developer interactions with coding agents.<n>We evaluate two leading copilot and agentic coding assistants.<n>Our results show agents can assist developers in ways that surpass copilots.
  arXiv  Detail & Related papers  (2025-07-10T20:12:54Z)
• Echoes of AI: Investigating the Downstream Effects of AI Assistants on Software Maintainability [5.677464428950146]
  This study investigates whether co-development with AI assistants affects software maintainability.<n> AI-assisted development in Phase 1 led to a modest speedup in subsequent evolution.<n>For habitual AI users, the mean speedup was 55.9%.
  arXiv  Detail & Related papers  (2025-07-01T14:24:37Z)
• On Developers' Self-Declaration of AI-Generated Code: An Analysis of Practices [2.205621813527632]
  This study aims to understand the ways developers use to self-declare AI-generated code.<n>In the first phase, we collected 613 instances of AI-generated code snippets from GitHub.<n>In the second phase, we conducted a follow-up practitioners' survey, which received 111 valid responses.
  arXiv  Detail & Related papers  (2025-04-23T07:52:39Z)
• Challenges and Paths Towards AI for Software Engineering [55.95365538122656]
  We discuss progress in AI for software engineering in threefold manner.<n>First, we provide a structured taxonomy of concrete tasks in AI for software engineering.<n>Second, we outline several key bottlenecks that limit current approaches.
  arXiv  Detail & Related papers  (2025-03-28T17:17:57Z)
• Almost AI, Almost Human: The Challenge of Detecting AI-Polished Writing [55.2480439325792]
  This study systematically evaluations twelve state-of-the-art AI-text detectors using our AI-Polished-Text Evaluation dataset.<n>Our findings reveal that detectors frequently flag even minimally polished text as AI-generated, struggle to differentiate between degrees of AI involvement, and exhibit biases against older and smaller models.
  arXiv  Detail & Related papers  (2025-02-21T18:45:37Z)
• Consent in Crisis: The Rapid Decline of the AI Data Commons [74.68176012363253]
  General-purpose artificial intelligence (AI) systems are built on massive swathes of public web data. We conduct the first, large-scale, longitudinal audit of the consent protocols for the web domains underlying AI training corpora.
  arXiv  Detail & Related papers  (2024-07-20T16:50:18Z)
• Generation Probabilities Are Not Enough: Uncertainty Highlighting in AI Code Completions [54.55334589363247]
  We study whether conveying information about uncertainty enables programmers to more quickly and accurately produce code. We find that highlighting tokens with the highest predicted likelihood of being edited leads to faster task completion and more targeted edits.
  arXiv  Detail & Related papers  (2023-02-14T18:43:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.