Understanding the Impact of AI Code Assistants on Security API Usage: An Empirical Study

• URL: http://arxiv.org/abs/2607.11348v1
• Date: Mon, 13 Jul 2026 10:08:24 GMT
• Title: Understanding the Impact of AI Code Assistants on Security API Usage: An Empirical Study
• Abstract summary: This study is the first empirical investigation into how AI code assistants affect professional developers' use of security APIs.<n>We conducted a study with 44 developers who completed security API programming tasks with and without GitHub Copilot assistance.<n>Our findings show that, while Copilot improves functional correctness and marginally reduces certain insecure patterns, it does not significantly improve secure API usage.
• Score: 7.44244145609835
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: AI code assistants are transforming software development, but their implications for software security remain a major concern, particularly in the context of security APIs. These APIs are critical for safeguarding software systems, yet their complexity often leads to incorrect use and serious vulnerabilities. Developing an evidence-based understanding of how AI assistants influence developers' use of these APIs is therefore essential for informing effective mitigation strategies. While a few user studies have examined the broader impact of AI assistants on software vulnerabilities, the use of security APIs remains unexplored from a developer-centered perspective. This study addresses this gap by presenting the first empirical investigation into how AI code assistants affect professional developers' use of security APIs. We conducted a study with 44 developers who completed security API programming tasks with and without GitHub Copilot assistance. Our findings show that, while Copilot improves functional correctness and marginally reduces certain insecure patterns, it does not significantly improve secure API usage. We also found that developers rarely raised security concerns when engaging with Copilot, and many did not recognize that their final implementations remained insecure. Finally, we offer recommendations for enhancing security awareness among developers and propose future research directions to support safer AI-assisted software development.

Related papers

• Understanding the (In)Security of Vibe-Coded Applications [3.954625565366601]
  vibe coding is an emerging software development paradigm in which users create applications primarily through natural-language interactions with AI agents.<n>Unlike conventional AI-assisted programming, vibe coding delegates a substantial portion of development to AI systems.<n>We collect a large corpus of real-world applications developed using popular AI agents and design a vulnerability analysis framework that combines agent-assisted code auditing with human validation.
  arXiv  Detail & Related papers  (2026-06-22T10:19:07Z)
• From Preventive to Reactive: How AI Coding Assistants Transform Developers' Security Awareness [3.2382588276702315]
  We conducted interviews with 15 professional software engineers and observed them completing security-relevant coding tasks with AI assistance.<n>We find that AI coding assistants reorganize rather than eliminate security thinking, shifting it from the act of writing code to the act of reviewing it.<n>This paper contributes a practice-grounded account of how AI-assisted development reshapes the human side of secure coding.
  arXiv  Detail & Related papers  (2026-05-22T01:07:15Z)
• ExploitGym: Can AI Agents Turn Security Vulnerabilities into Real Attacks? [92.21756459993695]
  Exploitation is a challenging task because it requires low-level program reasoning.<n>Despite its importance and diagnostic value, exploitation remains under-evaluated.<n>We introduce ExploitGym, a large-scale, diverse, realistic benchmark on the exploitation capabilities of AI agents.
  arXiv  Detail & Related papers  (2026-05-11T18:00:14Z)
• "We are not Future-ready": Understanding AI Privacy Risks and Existing Mitigation Strategies from the Perspective of AI Developers in Europe [56.1653658714305]
  We interviewed 25 AI developers based in Europe to understand which privacy threats they believe pose the greatest risk to users, developers, and businesses.<n>We find that there is little consensus among AI developers on the relative ranking of privacy risks.<n>While AI developers are aware of proposed mitigation strategies for addressing these risks, they reported minimal real-world adoption.
  arXiv  Detail & Related papers  (2025-10-01T13:51:33Z)
• GitHub's Copilot Code Review: Can AI Spot Security Flaws Before You Commit? [0.0]
  This study evaluates the effectiveness of GitHub Copilot's recently introduced code review feature in detecting security vulnerabilities.<n>Contrary to expectations, our results reveal that Copilot's code review frequently fails to detect critical vulnerabilities.<n>Our results highlight the continued necessity of dedicated security tools and manual code audits to ensure robust software security.
  arXiv  Detail & Related papers  (2025-09-17T02:56:21Z)
• Code with Me or for Me? How Increasing AI Automation Transforms Developer Workflows [60.04362496037186]
  We present the first controlled study of developer interactions with coding agents.<n>We evaluate two leading copilot and agentic coding assistants.<n>Our results show agents can assist developers in ways that surpass copilots.
  arXiv  Detail & Related papers  (2025-07-10T20:12:54Z)
• AISafetyLab: A Comprehensive Framework for AI Safety Evaluation and Improvement [73.0700818105842]
  We introduce AISafetyLab, a unified framework and toolkit that integrates representative attack, defense, and evaluation methodologies for AI safety.<n> AISafetyLab features an intuitive interface that enables developers to seamlessly apply various techniques.<n>We conduct empirical studies on Vicuna, analyzing different attack and defense strategies to provide valuable insights into their comparative effectiveness.
  arXiv  Detail & Related papers  (2025-02-24T02:11:52Z)
• SOK: Exploring Hallucinations and Security Risks in AI-Assisted Software Development with Insights for LLM Deployment [0.0]
  Large Language Models (LLMs) such as GitHub Copilot, ChatGPT, Cursor AI, and Codeium AI have revolutionized the coding landscape.<n>This paper provides a comprehensive analysis of the benefits and risks associated with AI-powered coding tools.
  arXiv  Detail & Related papers  (2025-01-31T06:00:27Z)
• Position: Mind the Gap-the Growing Disconnect Between Established Vulnerability Disclosure and AI Security [56.219994752894294]
  We argue that adapting existing processes for AI security reporting is doomed to fail due to fundamental shortcomings for the distinctive characteristics of AI systems.<n>Based on our proposal to address these shortcomings, we discuss an approach to AI security reporting and how the new AI paradigm, AI agents, will further reinforce the need for specialized AI security incident reporting advancements.
  arXiv  Detail & Related papers  (2024-12-19T13:50:26Z)
• Using AI Assistants in Software Development: A Qualitative Study on Security Practices and Concerns [23.867795468379743]
  Recent research has demonstrated that AI-generated code can contain security issues. How software professionals balance AI assistant usage and security remains unclear. This paper investigates how software professionals use AI assistants in secure software development.
  arXiv  Detail & Related papers  (2024-05-10T10:13:19Z)
• Proceedings of the Artificial Intelligence for Cyber Security (AICS) Workshop at AAAI 2022 [55.573187938617636]
  The workshop will focus on the application of AI to problems in cyber security. Cyber systems generate large volumes of data, utilizing this effectively is beyond human capabilities.
  arXiv  Detail & Related papers  (2022-02-28T18:27:41Z)
• Dos and Don'ts of Machine Learning in Computer Security [74.1816306998445]
  Despite great potential, machine learning in security is prone to subtle pitfalls that undermine its performance. We identify common pitfalls in the design, implementation, and evaluation of learning-based security systems. We propose actionable recommendations to support researchers in avoiding or mitigating the pitfalls where possible.
  arXiv  Detail & Related papers  (2020-10-19T13:09:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.