From Copilot to Pilot: Towards AI Supported Software Development

• URL: http://arxiv.org/abs/2303.04142v1
• Date: Tue, 7 Mar 2023 18:56:52 GMT
• Title: From Copilot to Pilot: Towards AI Supported Software Development
• Authors: Rohith Pudari, Neil A. Ernst
• Abstract summary: We study the limitations of AI-supported code completion tools like Copilot and offer a taxonomy for understanding the classification of AI-supported code completion tools in this space. We then conduct additional investigation to determine the current boundaries of AI-supported code completion tools like Copilot. We conclude by providing a discussion on challenges for future development of AI-supported code completion tools to reach the design level of abstraction in our taxonomy.
• Score: 3.0585424861188844
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: AI-supported programming has arrived, as shown by the introduction and successes of large language models for code, such as Copilot/Codex (Github/OpenAI) and AlphaCode (DeepMind). Above human average performance on programming challenges is now possible. However, software engineering is much more than solving programming contests. Moving beyond code completion to AI-supported software engineering will require an AI system that can, among other things, understand how to avoid code smells, to follow language idioms, and eventually (maybe!) propose rational software designs. In this study, we explore the current limitations of AI-supported code completion tools like Copilot and offer a simple taxonomy for understanding the classification of AI-supported code completion tools in this space. We first perform an exploratory study on Copilot's code suggestions for language idioms and code smells. Copilot does not follow language idioms and avoid code smells in most of our test scenarios. We then conduct additional investigation to determine the current boundaries of AI-supported code completion tools like Copilot by introducing a taxonomy of software abstraction hierarchies where 'basic programming functionality' such as code compilation and syntax checking is at the least abstract level, software architecture analysis and design are at the most abstract level. We conclude by providing a discussion on challenges for future development of AI-supported code completion tools to reach the design level of abstraction in our taxonomy.

Related papers

• Contextual Augmented Multi-Model Programming (CAMP): A Hybrid Local-Cloud Copilot Framework [8.28588489551341]
  This paper presents CAMP, a multi-model AI-assisted programming framework that consists of a local model that employs Retrieval-Augmented Generation (RAG) RAG retrieves contextual information from the cloud model to facilitate context-aware prompt construction. The methodology is actualized in Copilot for Xcode, an AI-assisted programming tool crafted for the Apple software ecosystem.
  arXiv  Detail & Related papers  (2024-10-20T04:51:24Z)
• RepoGraph: Enhancing AI Software Engineering with Repository-level Code Graph [63.87660059104077]
  We present RepoGraph, a plug-in module that manages a repository-level structure for modern AI software engineering solutions. RepoGraph substantially boosts the performance of all systems, leading to a new state-of-the-art among open-source frameworks.
  arXiv  Detail & Related papers  (2024-10-03T05:45:26Z)
• Automatic Programming: Large Language Models and Beyond [48.34544922560503]
  We study concerns around code quality, security and related issues of programmer responsibility. We discuss how advances in software engineering can enable automatic programming. We conclude with a forward looking view, focusing on the programming environment of the near future.
  arXiv  Detail & Related papers  (2024-05-03T16:19:24Z)
• Deep Learning for Code Intelligence: Survey, Benchmark and Toolkit [63.82016263181941]
  Code intelligence leverages machine learning techniques to extract knowledge from extensive code corpora. Currently, there is already a thriving research community focusing on code intelligence.
  arXiv  Detail & Related papers  (2023-12-30T17:48:37Z)
• Students' Perspective on AI Code Completion: Benefits and Challenges [2.936007114555107]
  We investigated the benefits, challenges, and expectations of AI code completion from students' perspectives. Our findings show that AI code completion enhanced students' productivity and efficiency by providing correct syntax suggestions. In the future, AI code completion should be explainable and provide best coding practices to enhance the education process.
  arXiv  Detail & Related papers  (2023-10-31T22:41:16Z)
• Copilot for Xcode: Exploring AI-Assisted Programming by Prompting Cloud-based Large Language Models [2.5272389610447856]
  Copilot for Xcode is an AI-assisted programming tool for program composition and design to support human software developers. By seamlessly integrating cloud-based Large Language Models (LLM) with Apple's local development environment, Xcode, this tool enhances productivity and unleashes creativity for software development in Apple software ecosystem.
  arXiv  Detail & Related papers  (2023-07-08T09:11:19Z)
• Natural Language Generation and Understanding of Big Code for AI-Assisted Programming: A Review [9.355153561673855]
  This paper focuses on transformer-based large language models (LLMs) trained using Big Code. LLMs have played a crucial role in facilitating AI-assisted programming applications, including code generation, code completion, code translation, code refinement, code summarization, defect detection, and clone detection. It explores the challenges and opportunities associated with incorporating NLP techniques with software naturalness in these applications.
  arXiv  Detail & Related papers  (2023-07-04T21:26:51Z)
• Generation Probabilities Are Not Enough: Uncertainty Highlighting in AI Code Completions [54.55334589363247]
  We study whether conveying information about uncertainty enables programmers to more quickly and accurately produce code. We find that highlighting tokens with the highest predicted likelihood of being edited leads to faster task completion and more targeted edits.
  arXiv  Detail & Related papers  (2023-02-14T18:43:34Z)
• Enabling Automated Machine Learning for Model-Driven AI Engineering [60.09869520679979]
  We propose a novel approach to enable Model-Driven Software Engineering and Model-Driven AI Engineering. In particular, we support Automated ML, thus assisting software engineers without deep AI knowledge in developing AI-intensive systems.
  arXiv  Detail & Related papers  (2022-03-06T10:12:56Z)
• Competition-Level Code Generation with AlphaCode [74.87216298566942]
  We introduce AlphaCode, a system for code generation that can create novel solutions to problems that require deeper reasoning. In simulated evaluations on recent programming competitions on the Codeforces platform, AlphaCode achieved on average a ranking of top 54.3%.
  arXiv  Detail & Related papers  (2022-02-08T23:16:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.