Factors Influencing the Quality of AI-Generated Code: A Synthesis of Empirical Evidence

• URL: http://arxiv.org/abs/2603.25146v1
• Date: Thu, 26 Mar 2026 08:06:42 GMT
• Title: Factors Influencing the Quality of AI-Generated Code: A Synthesis of Empirical Evidence
• Abstract summary: The rapid adoption of AI-assisted code generation tools is transforming software development practices.<n>Concerns regarding the quality, reliability, and security of AI-generated code are increasingly reported in both academia and industry.
• Score: 1.4465033892011254
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Context: The rapid adoption of AI-assisted code generation tools, such as large language models (LLMs), is transforming software development practices. While these tools promise significant productivity gains, concerns regarding the quality, reliability, and security of AI-generated code are increasingly reported in both academia and industry. --Objective: This study aims to systematically synthesize existing empirical evidence on the factors influencing the quality of AI-generated source code and to analyze how these factors impact software quality outcomes across different evaluation contexts. --Method: We conducted a systematic literature review (SLR) following established guidelines, supported by an AI-assisted workflow with human oversight. A total of 24 primary studies were selected through a structured search and screening process across major digital libraries. Data were extracted and analyzed using qualitative, pattern-based evidence synthesis. --Results: The findings reveal that code quality in AI-assisted development is influenced by a combination of human factors, AI system characteristics, and human AI interaction dynamics. Key influencing factors include prompt design, task specification, and developer expertise. The results also show variability in quality outcomes such as correctness, security, maintainability, and complexity across studies, with both improvements and risks reported. --Conclusion: AI-assisted code generation represents a socio-technical shift in software engineering, where achieving high-quality outcomes depends on both technological and human factors. While promising, AI-generated code requires careful validation and integration into development workflows.

Related papers

• How AI Coders Discuss, Disagree, and Reach Consensus: Challenges and Opportunities for LLM-Based Qualitative Coding [5.271725890500162]
  We develop a literature-informed baseline pipeline that enables AI agents to independently code, debate, and reconcile disagreements.<n>Results revealed that coding accuracy depends on factors such as codebook length, qualitative data similarity, and agent disagreement.<n>Our open-source AI discussion dataset and methodological framework lay the groundwork for advancing the design of AI-mediated automated thematic analysis.
  arXiv  Detail & Related papers  (2026-09-10T05:34:47Z)
• The Vibe Shift in Software Engineering: Evaluating AI-Led Conversational Programming for Performance, Cognition, and Responsible Adoption [0.0]
  Vibe Coding enables developers to generate software through natural-language interaction with large language models.<n>The study assessed performance efficiency, cognitive implications, and responsible adoption in comparison with traditional and AI-assisted coding environments.
  arXiv  Detail & Related papers  (2026-09-09T00:33:17Z)
• On Automated and Explainable Provenance of AI-Generated Code [56.818202904805446]
  We argue that the next generation of CodeGenAI tools must be built on a foundation of explainable provenance.<n>We describe a research program combining large-scale empirical studies with post-hoc causal and interpretability techniques.
  arXiv  Detail & Related papers  (2026-08-03T14:50:31Z)
• A Survey of Vibe Coding with Large Language Models [93.88284590533242]
  "Vibe Coding" is a development methodology where developers validate AI-generated implementations through outcome observation.<n>Despite its transformative potential, the effectiveness of this emergent paradigm remains under-explored.<n>This survey provides the first comprehensive and systematic review of Vibe Coding with large language models.
  arXiv  Detail & Related papers  (2025-10-14T11:26:56Z)
• Towards the Next Generation of Software: Insights from Grey Literature on AI-Native Applications [13.876049229274114]
  AI-native applications are a new paradigm in software engineering that fundamentally redefines how software is designed, developed, and evolved.<n>Despite their growing prominence, AI-native applications still lack a unified engineering definition and architectural blueprint.<n>This study seeks to establish a comprehensive understanding of AI-native applications by identifying their defining characteristics, key quality attributes, and typical technology stacks.
  arXiv  Detail & Related papers  (2025-09-16T15:01:23Z)
• The AI Imperative: Scaling High-Quality Peer Review in Machine Learning [49.87236114682497]
  We argue that AI-assisted peer review must become an urgent research and infrastructure priority.<n>We propose specific roles for AI in enhancing factual verification, guiding reviewer performance, assisting authors in quality improvement, and supporting ACs in decision-making.
  arXiv  Detail & Related papers  (2025-06-09T18:37:14Z)
• When Models Know More Than They Can Explain: Quantifying Knowledge Transfer in Human-AI Collaboration [79.69935257008467]
  We introduce Knowledge Integration and Transfer Evaluation (KITE), a conceptual and experimental framework for Human-AI knowledge transfer capabilities.<n>We conduct the first large-scale human study (N=118) explicitly designed to measure it.<n>In our two-phase setup, humans first ideate with an AI on problem-solving strategies, then independently implement solutions, isolating model explanations' influence on human understanding.
  arXiv  Detail & Related papers  (2025-06-05T20:48:16Z)
• Human-AI Experience in Integrated Development Environments: A Systematic Literature Review [2.1749194587826026]
  The integration of Artificial Intelligence into Integrated Development Environments (IDEs) is reshaping how developers interact with their tools.<n>This shift marks the emergence of Human-AI Experience in Integrated Development Environment (in-IDE HAX)<n>Research on in-IDE HAX remains fragmented, which highlights the need for a unified overview of current practices, challenges, and opportunities.
  arXiv  Detail & Related papers  (2025-03-08T12:40:18Z)
• Computational Safety for Generative AI: A Signal Processing Perspective [65.268245109828]
  computational safety is a mathematical framework that enables the quantitative assessment, formulation, and study of safety challenges in GenAI.<n>We show how sensitivity analysis and loss landscape analysis can be used to detect malicious prompts with jailbreak attempts.<n>We discuss key open research challenges, opportunities, and the essential role of signal processing in computational AI safety.
  arXiv  Detail & Related papers  (2025-02-18T02:26:50Z)
• Data Analysis in the Era of Generative AI [56.44807642944589]
  This paper explores the potential of AI-powered tools to reshape data analysis, focusing on design considerations and challenges. We explore how the emergence of large language and multimodal models offers new opportunities to enhance various stages of data analysis workflow. We then examine human-centered design principles that facilitate intuitive interactions, build user trust, and streamline the AI-assisted analysis workflow across multiple apps.
  arXiv  Detail & Related papers  (2024-09-27T06:31:03Z)
• Comparing Software Developers with ChatGPT: An Empirical Investigation [0.0]
  This paper conducts an empirical investigation, contrasting the performance of software engineers and AI systems, like ChatGPT, across different evaluation metrics. The paper posits that a comprehensive comparison of software engineers and AI-based solutions, considering various evaluation criteria, is pivotal in fostering human-machine collaboration.
  arXiv  Detail & Related papers  (2023-05-19T17:25:54Z)
• What is Software Quality for AI Engineers? Towards a Thinning of the Fog [9.401273164668092]
  The goal of this study is to investigate the software quality assurance strategies adopted during the development, integration, and maintenance of AI/ML components and code. A qualitative analysis of the interview data identified 12 issues in the development of AI/ML components. The results of this study should guide future work on software quality assurance processes and techniques for AI/ML components.
  arXiv  Detail & Related papers  (2022-03-23T19:43:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.