The Impact of AI-Assisted Development on Software Security: A Study of Gemini and Developer Experience

• URL: http://arxiv.org/abs/2603.15298v2
• Date: Tue, 17 Mar 2026 10:48:08 GMT
• Title: The Impact of AI-Assisted Development on Software Security: A Study of Gemini and Developer Experience
• Abstract summary: This study explores the impact of Google's AI tool Gemini on code security.<n> participants were assigned a security-related programming task using either no AI tools, the free version, or the paid version of Gemini.<n>While we did not observe significant differences between using Gemini in terms of secure software development, programming experience significantly improved code security and cannot be fully substituted by Gemini.
• Score: 14.378041471593315
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The ongoing shortage of skilled developers, particularly in security-critical software development, has led organizations to increasingly adopt AI-powered development tools to boost productivity and reduce reliance on limited human expertise. These tools, often based on large language models, aim to automate routine tasks and make secure software development more accessible and efficient. However, it remains unclear how developers' general programming and security-specific experience, and the type of AI tool used (free vs. paid) affect the security of the resulting software. Therefore, we conducted a quantitative programming study with software developers (n=159) exploring the impact of Google's AI tool Gemini on code security. Participants were assigned a security-related programming task using either no AI tools, the free version, or the paid version of Gemini. While we did not observe significant differences between using Gemini in terms of secure software development, programming experience significantly improved code security and cannot be fully substituted by Gemini.

Related papers

• Understanding the Impact of AI Code Assistants on Security API Usage: An Empirical Study [7.44244145609835]
  This study is the first empirical investigation into how AI code assistants affect professional developers' use of security APIs.<n>We conducted a study with 44 developers who completed security API programming tasks with and without GitHub Copilot assistance.<n>Our findings show that, while Copilot improves functional correctness and marginally reduces certain insecure patterns, it does not significantly improve secure API usage.
  arXiv  Detail & Related papers  (2026-07-13T10:08:24Z)
• ExploitGym: Can AI Agents Turn Security Vulnerabilities into Real Attacks? [92.21756459993695]
  Exploitation is a challenging task because it requires low-level program reasoning.<n>Despite its importance and diagnostic value, exploitation remains under-evaluated.<n>We introduce ExploitGym, a large-scale, diverse, realistic benchmark on the exploitation capabilities of AI agents.
  arXiv  Detail & Related papers  (2026-05-11T18:00:14Z)
• SEAgent: Self-Evolving Computer Use Agent with Autonomous Learning from Experience [71.82719117238307]
  We propose SEAgent, an agentic self-evolving framework enabling computer-use agents to evolve through interactions with unfamiliar software.<n>We validate the effectiveness of SEAgent across five novel software environments within OS-World.<n>Our approach achieves a significant improvement of 23.2% in success rate, from 11.3% to 34.5%, over a competitive open-source CUA.
  arXiv  Detail & Related papers  (2025-08-06T17:58:46Z)
• Code with Me or for Me? How Increasing AI Automation Transforms Developer Workflows [60.04362496037186]
  We present the first controlled study of developer interactions with coding agents.<n>We evaluate two leading copilot and agentic coding assistants.<n>Our results show agents can assist developers in ways that surpass copilots.
  arXiv  Detail & Related papers  (2025-07-10T20:12:54Z)
• The Role of Generative AI in Strengthening Secure Software Coding Practices: A Systematic Perspective [0.0]
  The integration of Generative AI (GenAI) into software development holds significant potential for improving secure coding practices.<n>This paper aims at systematically studying the impact of GenAI in enhancing secure coding practices.
  arXiv  Detail & Related papers  (2025-04-28T04:01:12Z)
• AISafetyLab: A Comprehensive Framework for AI Safety Evaluation and Improvement [73.0700818105842]
  We introduce AISafetyLab, a unified framework and toolkit that integrates representative attack, defense, and evaluation methodologies for AI safety.<n> AISafetyLab features an intuitive interface that enables developers to seamlessly apply various techniques.<n>We conduct empirical studies on Vicuna, analyzing different attack and defense strategies to provide valuable insights into their comparative effectiveness.
  arXiv  Detail & Related papers  (2025-02-24T02:11:52Z)
• SOK: Exploring Hallucinations and Security Risks in AI-Assisted Software Development with Insights for LLM Deployment [0.0]
  Large Language Models (LLMs) such as GitHub Copilot, ChatGPT, Cursor AI, and Codeium AI have revolutionized the coding landscape.<n>This paper provides a comprehensive analysis of the benefits and risks associated with AI-powered coding tools.
  arXiv  Detail & Related papers  (2025-01-31T06:00:27Z)
• "I Don't Use AI for Everything": Exploring Utility, Attitude, and Responsibility of AI-empowered Tools in Software Development [19.851794567529286]
  This study investigates the adoption, impact, and security considerations of AI-empowered tools in the software development process. Our findings reveal widespread adoption of AI tools across various stages of software development.
  arXiv  Detail & Related papers  (2024-09-20T09:17:10Z)
• Using AI Assistants in Software Development: A Qualitative Study on Security Practices and Concerns [23.867795468379743]
  Recent research has demonstrated that AI-generated code can contain security issues. How software professionals balance AI assistant usage and security remains unclear. This paper investigates how software professionals use AI assistants in secure software development.
  arXiv  Detail & Related papers  (2024-05-10T10:13:19Z)
• Embedded Software Development with Digital Twins: Specific Requirements for Small and Medium-Sized Enterprises [55.57032418885258]
  Digital twins have the potential for cost-effective software development and maintenance strategies. We interviewed SMEs about their current development processes. First results show that real-time requirements prevent, to date, a Software-in-the-Loop development approach.
  arXiv  Detail & Related papers  (2023-09-17T08:56:36Z)
• Generation Probabilities Are Not Enough: Uncertainty Highlighting in AI Code Completions [54.55334589363247]
  We study whether conveying information about uncertainty enables programmers to more quickly and accurately produce code. We find that highlighting tokens with the highest predicted likelihood of being edited leads to faster task completion and more targeted edits.
  arXiv  Detail & Related papers  (2023-02-14T18:43:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.