What is it like to program with artificial intelligence?
- URL: http://arxiv.org/abs/2208.06213v1
- Date: Fri, 12 Aug 2022 10:48:46 GMT
- Title: What is it like to program with artificial intelligence?
- Authors: Advait Sarkar, Andrew D. Gordon, Carina Negreanu, Christian Poelitz,
Sruti Srinivasa Ragavan, Ben Zorn
- Abstract summary: Large language models can generate code to solve a variety of problems expressed in natural language.
This technology has already been commercialised in at least one widely-used programming editor extension: GitHub Copilot.
We explore how programming with large language models (LLM-assisted programming) is similar to, and differs from, prior conceptualisations of programmer assistance.
- Score: 10.343988028594612
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Large language models, such as OpenAI's codex and Deepmind's AlphaCode, can
generate code to solve a variety of problems expressed in natural language.
This technology has already been commercialised in at least one widely-used
programming editor extension: GitHub Copilot.
In this paper, we explore how programming with large language models
(LLM-assisted programming) is similar to, and differs from, prior
conceptualisations of programmer assistance. We draw upon publicly available
experience reports of LLM-assisted programming, as well as prior usability and
design studies. We find that while LLM-assisted programming shares some
properties of compilation, pair programming, and programming via search and
reuse, there are fundamental differences both in the technical possibilities as
well as the practical experience. Thus, LLM-assisted programming ought to be
viewed as a new way of programming with its own distinct properties and
challenges.
Finally, we draw upon observations from a user study in which non-expert end
user programmers use LLM-assisted tools for solving data tasks in spreadsheets.
We discuss the issues that might arise, and open research challenges, in
applying large language models to end-user programming, particularly with users
who have little or no programming expertise.
Related papers
- AIOS Compiler: LLM as Interpreter for Natural Language Programming and Flow Programming of AI Agents [38.580779075892636]
We develop a novel system for Code Representation and Execution (CoRE)
The proposed system unifies natural language programming, pseudo-code programming, and flow programming under the same representation for constructing language agents.
During the execution, we incorporate external memory to minimize redundancy.
arXiv Detail & Related papers (2024-05-11T04:29:03Z) - Automatic Programming: Large Language Models and Beyond [48.34544922560503]
We study concerns around code quality, security and related issues of programmer responsibility.
We discuss how advances in software engineering can enable automatic programming.
We conclude with a forward looking view, focusing on the programming environment of the near future.
arXiv Detail & Related papers (2024-05-03T16:19:24Z) - CodeGRAG: Bridging the Gap between Natural Language and Programming Language via Graphical Retrieval Augmented Generation [58.84212778960507]
We propose CodeGRAG, a Graphical Retrieval Augmented Code Generation framework to enhance the performance of LLMs.
CodeGRAG builds the graphical view of code blocks based on the control flow and data flow of them to fill the gap between programming languages and natural language.
Various experiments and ablations are done on four datasets including both the C++ and python languages to validate the hard meta-graph prompt, the soft prompting technique, and the effectiveness of the objectives for pretrained GNN expert.
arXiv Detail & Related papers (2024-05-03T02:48:55Z) - CSEPrompts: A Benchmark of Introductory Computer Science Prompts [11.665831944836118]
Recent advances in AI, machine learning, and NLP have led to the development of a new generation of Large Language Models (LLMs)
Commercial applications have made this technology available to the general public, thus making it possible to use LLMs to produce high-quality texts for academic and professional purposes.
Schools and universities are aware of the increasing use of AI-generated content by students and they have been researching the impact of this new technology and its potential misuse.
arXiv Detail & Related papers (2024-04-03T07:55:57Z) - PwR: Exploring the Role of Representations in Conversational Programming [17.838776812138626]
We introduce Programming with Representations (PwR), an approach that uses representations to convey the system's understanding back to the user in natural language.
We find that representations significantly improve understandability, and instilled a sense of agency among our participants.
arXiv Detail & Related papers (2023-09-18T05:38:23Z) - Natural Language Generation and Understanding of Big Code for
AI-Assisted Programming: A Review [9.355153561673855]
This paper focuses on transformer-based large language models (LLMs) trained using Big Code.
LLMs have played a crucial role in facilitating AI-assisted programming applications, including code generation, code completion, code translation, code refinement, code summarization, defect detection, and clone detection.
It explores the challenges and opportunities associated with incorporating NLP techniques with software naturalness in these applications.
arXiv Detail & Related papers (2023-07-04T21:26:51Z) - CodeTF: One-stop Transformer Library for State-of-the-art Code LLM [72.1638273937025]
We present CodeTF, an open-source Transformer-based library for state-of-the-art Code LLMs and code intelligence.
Our library supports a collection of pretrained Code LLM models and popular code benchmarks.
We hope CodeTF is able to bridge the gap between machine learning/generative AI and software engineering.
arXiv Detail & Related papers (2023-05-31T05:24:48Z) - CREATOR: Tool Creation for Disentangling Abstract and Concrete Reasoning of Large Language Models [74.22729793816451]
Large Language Models (LLMs) have made significant progress in utilizing tools, but their ability is limited by API availability.
We propose CREATOR, a novel framework that enables LLMs to create their own tools using documentation and code realization.
We evaluate CREATOR on MATH and TabMWP benchmarks, respectively consisting of challenging math competition problems.
arXiv Detail & Related papers (2023-05-23T17:51:52Z) - Competition-Level Code Generation with AlphaCode [74.87216298566942]
We introduce AlphaCode, a system for code generation that can create novel solutions to problems that require deeper reasoning.
In simulated evaluations on recent programming competitions on the Codeforces platform, AlphaCode achieved on average a ranking of top 54.3%.
arXiv Detail & Related papers (2022-02-08T23:16:31Z) - Natural Language-guided Programming [1.3955252961896318]
We put forward a vision based on a new breed of developer tools that have the potential to largely automate this process.
Key idea is to adapt code autocompletion tools such that they take into account not only the developer's already-written code but also the intent of the task the developer is trying to achieve next.
We call this practice of enriching the code with natural language intent to facilitate its completion natural language-guided programming.
arXiv Detail & Related papers (2021-08-11T13:06:33Z) - Leveraging Language to Learn Program Abstractions and Search Heuristics [66.28391181268645]
We introduce LAPS (Language for Abstraction and Program Search), a technique for using natural language annotations to guide joint learning of libraries and neurally-guided search models for synthesis.
When integrated into a state-of-the-art library learning system (DreamCoder), LAPS produces higher-quality libraries and improves search efficiency and generalization.
arXiv Detail & Related papers (2021-06-18T15:08:47Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.