Usage, Effects and Requirements for AI Coding Assistants in the Enterprise: An Empirical Study

• URL: http://arxiv.org/abs/2601.20112v1
• Date: Tue, 27 Jan 2026 22:57:20 GMT
• Title: Usage, Effects and Requirements for AI Coding Assistants in the Enterprise: An Empirical Study
• Authors: Maja Vukovic, Rangeet Pan, Tin Kam Ho, Rahul Krishna, Raju Pavuluri, Michele Merler,
• Abstract summary: We surveyed 57 developers about their experience with AI coding assistants and CodeLLMs.<n>We reviewed 35 user surveys on the usage, experience and expectations of professionals and students using AI coding assistants and CodeLLMs.
• Score: 4.01226690413941
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The rise of large language models (LLMs) has accelerated the development of automated techniques and tools for supporting various software engineering tasks, e.g., program understanding, code generation, software testing, and program repair. As CodeLLMs are being employed toward automating these tasks, one question that arises, especially in enterprise settings, is whether these coding assistants and the code LLMs that power them are ready for real-world projects and enterprise use cases, and how do they impact the existing software engineering process and user experience. In this paper we survey 57 developers from different domains and with varying software engineering skill about their experience with AI coding assistants and CodeLLMs. We also reviewed 35 user surveys on the usage, experience and expectations of professionals and students using AI coding assistants and CodeLLMs. Based on our study findings and analysis of existing surveys, we discuss the requirements for AI-powered coding assistants.

Related papers

• Learning to Code with Context: A Study-Based Approach [0.28675177318965045]
  The rapid emergence of generative AI tools is transforming the way software is developed.<n>Project-based courses offer an effective environment to explore and evaluate the integration of AI assistance into real-world development practices.<n>This paper presents our approach and a user study conducted within a university programming project in which students collaboratively developed computer games.
  arXiv  Detail & Related papers  (2025-12-04T20:40:36Z)
• From Code Foundation Models to Agents and Applications: A Practical Guide to Code Intelligence [150.3696990310269]
  Large language models (LLMs) have transformed automated software development by enabling direct translation of natural language descriptions into functional code.<n>We provide a comprehensive synthesis and practical guide (a series of analytic and probing experiments) about code LLMs.<n>We analyze the code capability of the general LLMs (GPT-4, Claude, LLaMA) and code-specialized LLMs (StarCoder, Code LLaMA, DeepSeek-Coder, and QwenCoder)
  arXiv  Detail & Related papers  (2025-11-23T17:09:34Z)
• Using LLMs and Essence to Support Software Practice Adoption [0.3609538870261841]
  This study explores the integration of Essence, a standard and thinking framework for managing software engineering practices, with large language models (LLMs)<n>The proposed system consistently outperforms its baseline counterpart in domain-specific tasks.
  arXiv  Detail & Related papers  (2025-08-22T14:59:35Z)
• A Survey on Code Generation with LLM-based Agents [61.474191493322415]
  Code generation agents powered by large language models (LLMs) are revolutionizing the software development paradigm.<n>LLMs are characterized by three core features.<n>This paper presents a systematic survey of the field of LLM-based code generation agents.
  arXiv  Detail & Related papers  (2025-07-31T18:17:36Z)
• Do AI models help produce verified bug fixes? [62.985237003585674]
  Large Language Models are used to produce corrections to software bugs.<n>This paper investigates how programmers use Large Language Models to complement their own skills.<n>The results are a first step towards a proper role for AI and LLMs in providing guaranteed-correct fixes to program bugs.
  arXiv  Detail & Related papers  (2025-07-21T17:30:16Z)
• Code with Me or for Me? How Increasing AI Automation Transforms Developer Workflows [60.04362496037186]
  We present the first controlled study of developer interactions with coding agents.<n>We evaluate two leading copilot and agentic coding assistants.<n>Our results show agents can assist developers in ways that surpass copilots.
  arXiv  Detail & Related papers  (2025-07-10T20:12:54Z)
• Examining the Use and Impact of an AI Code Assistant on Developer Productivity and Experience in the Enterprise [16.33294877731443]
  watsonx Code Assistant (WCA) deployed internally within IBM.<n>We examined developers' experiences with WCA and its impact on their productivity.
  arXiv  Detail & Related papers  (2024-12-09T15:53:00Z)
• Agent-Driven Automatic Software Improvement [55.2480439325792]
  This research proposal aims to explore innovative solutions by focusing on the deployment of agents powered by Large Language Models (LLMs) The iterative nature of agents, which allows for continuous learning and adaptation, can help surpass common challenges in code generation. We aim to use the iterative feedback in these systems to further fine-tune the LLMs underlying the agents, becoming better aligned to the task of automated software improvement.
  arXiv  Detail & Related papers  (2024-06-24T15:45:22Z)
• Making Software Development More Diverse and Inclusive: Key Themes, Challenges, and Future Directions [50.545824691484796]
  We identify six themes around the theme challenges and opportunities to improve Software Developer Diversity and Inclusion (SDDI)<n>We identify benefits, harms, and future research directions for the four main themes.<n>We discuss the remaining two themes, Artificial Intelligence & SDDI and AI & Computer Science education, which have a cross-cutting effect on the other themes.
  arXiv  Detail & Related papers  (2024-04-10T16:18:11Z)
• Developer Experiences with a Contextualized AI Coding Assistant: Usability, Expectations, and Outcomes [11.520721038793285]
  This study focuses on the initial experiences of 62 participants who used a contextualized coding AI assistant -- named StackSpot AI -- in a controlled setting. Assistants' use resulted in significant time savings, easier access to documentation, and the generation of accurate codes for internal APIs. challenges associated with the knowledge sources necessary to make the coding assistant access more contextual information as well as variable responses and limitations in handling complex codes were observed.
  arXiv  Detail & Related papers  (2023-11-30T10:52:28Z)
• Automated Machine Learning: A Case Study on Non-Intrusive Appliance Load Monitoring [81.06807079998117]
  We propose a novel approach to enable Automated Machine Learning (AutoML) for Non-Intrusive Appliance Load Monitoring (NIALM)<n>NIALM offers a cost-effective alternative to smart meters for measuring the energy consumption of electric devices and appliances.
  arXiv  Detail & Related papers  (2022-03-06T10:12:56Z)
• Software engineering for artificial intelligence and machine learning software: A systematic literature review [6.681725960709127]
  This study aims to investigate how software engineering has been applied in the development of AI/ML systems. Main challenges faced by professionals are in areas of testing, AI software quality, and data management.
  arXiv  Detail & Related papers  (2020-11-07T11:06:28Z)
• Machine Learning for Software Engineering: A Systematic Mapping [73.30245214374027]
  The software development industry is rapidly adopting machine learning for transitioning modern day software systems towards highly intelligent and self-learning systems. No comprehensive study exists that explores the current state-of-the-art on the adoption of machine learning across software engineering life cycle stages. This study introduces a machine learning for software engineering (MLSE) taxonomy classifying the state-of-the-art machine learning techniques according to their applicability to various software engineering life cycle stages.
  arXiv  Detail & Related papers  (2020-05-27T11:56:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.