Beyond Functional Correctness: Design Issues in AI IDE-Generated Large-Scale Projects

• URL: http://arxiv.org/abs/2604.06373v1
• Date: Tue, 07 Apr 2026 18:59:05 GMT
• Title: Beyond Functional Correctness: Design Issues in AI IDE-Generated Large-Scale Projects
• Abstract summary: We show that Cursor can generate functional large-scale projects averaging 16,965 LoC and 114 files.<n>The generated projects nevertheless contain design issues that may pose long-term maintainability and evolvability risks.
• Score: 5.428703898705396
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: New generation of AI coding tools, including AI-powered IDEs equipped with agentic capabilities, can generate code within the context of the project. These AI IDEs are increasingly perceived as capable of producing project-level code at scale. However, there is limited empirical evidence on the extent to which they can generate large-scale software systems and what design issues such systems may exhibit. To address this gap, we conducted a study to explore the capability of Cursor in generating large-scale projects and to evaluate the design quality of projects generated by Cursor. First, we propose a Feature-Driven Human-In-The-Loop (FD-HITL) framework that systematically guides project generation from curated project descriptions. We generated 10 projects using Cursor with the FD-HITL framework across three application domains and multiple technologies. We assessed the functional correctness of these projects through manual evaluation, obtaining an average functional correctness score of 91%. Next, we analyzed the generated projects using two static analysis tools, CodeScene and SonarQube, to detect design issues. We identified 1,305 design issues categorized into 9 categories by CodeScene and 3,193 issues in 11 categories by SonarQube. Our findings show that (1) when used with the FD-HITL framework, Cursor can generate functional large-scale projects averaging 16,965 LoC and 114 files; (2) the generated projects nevertheless contain design issues that may pose long-term maintainability and evolvability risks, requiring careful review by experienced developers; (3) the most prevalent issues include Code Duplication, high Code Complexity, Large Methods, Framework Best-Practice Violations, Exception-Handling Issues and Accessibility Issues; (4) these design issues violate design principles such as SRP, SoC, and DRY. The replication package is at https://github.com/Kashifraz/DIinAGP

Related papers

• Engineering Pitfalls in AI Coding Tools: An Empirical Study of Bugs in Claude Code, Codex, and Gemini CLI [14.169228579189989]
  This paper presents the first empirical study of engineering pitfalls in building AI-assisted coding tools.<n>We employ an open-coding methodology to analyze over 3.8K publicly reported bugs in the open-source repositories of three AI-assisted coding tools on GitHub.<n>Our results show that more than 67% of the bugs in these tools are related to functionality.
  arXiv  Detail & Related papers  (2026-03-21T15:05:23Z)
• IQuest-Coder-V1 Technical Report [56.045533451719905]
  We introduce the IQuest-Coder-V1 series-(7B/14B/40B/40B-Loop), a new family of code large language models (LLMs)<n>IQuest-Coder-V1 achieves state-of-the-art performance among competitive models across critical dimensions of code intelligence.
  arXiv  Detail & Related papers  (2026-03-17T16:15:31Z)
• ProjDevBench: Benchmarking AI Coding Agents on End-to-End Project Development [49.63491095660809]
  ProjDevBench is an end-to-end benchmark that provides project requirements to coding agents and evaluates the resulting repositories.<n>We curate 20 programming problems across 8 categories, covering both concept-oriented tasks and real-world application scenarios.<n>Our evaluation reports an overall acceptance rate of 27.38%: agents handle basic functionality but struggle with complex system design, time optimization, and resource management.
  arXiv  Detail & Related papers  (2026-02-02T05:17:23Z)
• An Empirical Study of Developer-Provided Context for AI Coding Assistants in Open-Source Projects [2.392035679895744]
  This paper presents a large-scale empirical study to characterize the emerging form of developer-provided context.<n>We developed a comprehensive taxonomy of project context that developers consider essential, organized into five high-level themes.<n>Our study also explores how this context varies across different project types and programming languages.
  arXiv  Detail & Related papers  (2025-12-21T23:51:02Z)
• Towards Realistic Project-Level Code Generation via Multi-Agent Collaboration and Semantic Architecture Modeling [7.753074942497876]
  We introduce CodeProjectEval, a project-level code generation dataset built from 18 real-world repositories with 12.7 files and 2,388.6 lines of code per task.<n>We propose ProjectGen, a multi-agent framework that decomposes projects into architecture design, skeleton generation, and code filling stages.<n>Experiments show that ProjectGen achieves state-of-the-art performance, passing 52/124 test cases on the small-scale project-level code generation dataset DevBench.
  arXiv  Detail & Related papers  (2025-11-05T12:12:35Z)
• Knowledge-Guided Multi-Agent Framework for Application-Level Software Code Generation [47.691865459572995]
  This paper envisions a Knowledge-Guided Application-Level Code Generation framework named KGACG.<n> KGACG aims to trans- form software requirements specification and architectural design document into executable code.
  arXiv  Detail & Related papers  (2025-10-22T03:10:58Z)
• A Survey on Code Generation with LLM-based Agents [61.474191493322415]
  Code generation agents powered by large language models (LLMs) are revolutionizing the software development paradigm.<n>LLMs are characterized by three core features.<n>This paper presents a systematic survey of the field of LLM-based code generation agents.
  arXiv  Detail & Related papers  (2025-07-31T18:17:36Z)
• CXXCrafter: An LLM-Based Agent for Automated C/C++ Open Source Software Building [14.687126587793028]
  C/C++ projects often proves to be difficult in practice, hindering the progress of downstream applications.<n>We develop an automated build system called CXXCrafter to address the challenges, such as dependency resolution.<n>Our evaluation on open-source software demonstrates that CXXCrafter achieves a success rate of 78% in project building.
  arXiv  Detail & Related papers  (2025-05-27T11:54:56Z)
• Towards Exception Safety Code Generation with Intermediate Representation Agents Framework [54.03528377384397]
  Large Language Models (LLMs) often struggle with robust exception handling in generated code, leading to fragile programs that are prone to runtime errors.<n>We propose Seeker, a novel multi-agent framework that enforces exception safety in LLM generated code through an Intermediate Representation (IR) approach.<n>Seeker decomposes exception handling into five specialized agents: Scanner, Detector, Predator, Ranker, and Handler.
  arXiv  Detail & Related papers  (2024-10-09T14:45:45Z)
• DevEval: Evaluating Code Generation in Practical Software Projects [52.16841274646796]
  We propose a new benchmark named DevEval, aligned with Developers' experiences in practical projects. DevEval is collected through a rigorous pipeline, containing 2,690 samples from 119 practical projects. We assess five popular LLMs on DevEval and reveal their actual abilities in code generation.
  arXiv  Detail & Related papers  (2024-01-12T06:51:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.