AI for Low-Code for AI
- URL: http://arxiv.org/abs/2305.20015v1
- Date: Wed, 31 May 2023 16:44:03 GMT
- Title: AI for Low-Code for AI
- Authors: Nikitha Rao, Jason Tsay, Kiran Kate, Vincent J. Hellendoorn, Martin
Hirzel
- Abstract summary: LowCoder is the first low-code tool for developing AI pipelines that supports both a visual programming interface and an AI-powered natural language interface.
We task 20 developers with varying levels of AI expertise with implementing four ML pipelines using LowCoder.
We find that LowCoder is especially useful for (i) Discoverability: using LowCoder_NL, participants discovered new operators in 75% of the tasks.
- Score: 8.379047663193422
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Low-code programming allows citizen developers to create programs with
minimal coding effort, typically via visual (e.g. drag-and-drop) interfaces. In
parallel, recent AI-powered tools such as Copilot and ChatGPT generate programs
from natural language instructions. We argue that these modalities are
complementary: tools like ChatGPT greatly reduce the need to memorize large
APIs but still require their users to read (and modify) programs, whereas
visual tools abstract away most or all programming but struggle to provide easy
access to large APIs. At their intersection, we propose LowCoder, the first
low-code tool for developing AI pipelines that supports both a visual
programming interface (LowCoder_VP) and an AI-powered natural language
interface (LowCoder_NL). We leverage this tool to provide some of the first
insights into whether and how these two modalities help programmers by
conducting a user study. We task 20 developers with varying levels of AI
expertise with implementing four ML pipelines using LowCoder, replacing the
LowCoder_NL component with a simple keyword search in half the tasks. Overall,
we find that LowCoder is especially useful for (i) Discoverability: using
LowCoder_NL, participants discovered new operators in 75% of the tasks,
compared to just 32.5% and 27.5% using web search or scrolling through options
respectively in the keyword-search condition, and (ii) Iterative Composition:
82.5% of tasks were successfully completed and many initial pipelines were
further successfully improved. Qualitative analysis shows that AI helps users
discover how to implement constructs when they know what to do, but still fails
to support novices when they lack clarity on what they want to accomplish.
Overall, our work highlights the benefits of combining the power of AI with
low-code programming.
Related papers
- Using AI-Based Coding Assistants in Practice: State of Affairs, Perceptions, and Ways Forward [9.177785129949]
We carried out a large-scale survey aimed at how AI assistants are used.
We collected opinions of 481 programmers on five broad activities.
Our results show that usage of AI assistants varies depending on activity and stage.
arXiv Detail & Related papers (2024-06-11T23:10:43Z) - How Far Have We Gone in Stripped Binary Code Understanding Using Large Language Models [51.527805834378974]
We propose a benchmark to evaluate the effectiveness of Large Language Models (LLMs) in binary code understanding.
Our evaluations reveal that existing LLMs can understand binary code to a certain extent, thereby improving the efficiency of binary code analysis.
arXiv Detail & Related papers (2024-04-15T14:44:08Z) - Genetic Auto-prompt Learning for Pre-trained Code Intelligence Language Models [54.58108387797138]
We investigate the effectiveness of prompt learning in code intelligence tasks.
Existing automatic prompt design methods are very limited to code intelligence tasks.
We propose Genetic Auto Prompt (GenAP) which utilizes an elaborate genetic algorithm to automatically design prompts.
arXiv Detail & Related papers (2024-03-20T13:37:00Z) - Generating Java Methods: An Empirical Assessment of Four AI-Based Code
Assistants [5.32539007352208]
We assess the effectiveness of four popular AI-based code assistants, namely GitHub Copilot, Tabnine, ChatGPT, and Google Bard.
Results show that Copilot is often more accurate than other techniques, yet none of the assistants is completely subsumed by the rest of the approaches.
arXiv Detail & Related papers (2024-02-13T12:59:20Z) - Developer Experiences with a Contextualized AI Coding Assistant:
Usability, Expectations, and Outcomes [11.520721038793285]
This study focuses on the initial experiences of 62 participants who used a contextualized coding AI assistant -- named StackSpot AI -- in a controlled setting.
Assistants' use resulted in significant time savings, easier access to documentation, and the generation of accurate codes for internal APIs.
challenges associated with the knowledge sources necessary to make the coding assistant access more contextual information as well as variable responses and limitations in handling complex codes were observed.
arXiv Detail & Related papers (2023-11-30T10:52:28Z) - Guess & Sketch: Language Model Guided Transpilation [59.02147255276078]
Learned transpilation offers an alternative to manual re-writing and engineering efforts.
Probabilistic neural language models (LMs) produce plausible outputs for every input, but do so at the cost of guaranteed correctness.
Guess & Sketch extracts alignment and confidence information from features of the LM then passes it to a symbolic solver to resolve semantic equivalence.
arXiv Detail & Related papers (2023-09-25T15:42:18Z) - A Large-Scale Survey on the Usability of AI Programming Assistants:
Successes and Challenges [23.467373994306524]
In practice, developers do not accept AI programming assistants' initial suggestions at a high frequency.
To understand developers' practices while using these tools, we administered a survey to a large population of developers.
We found that developers are most motivated to use AI programming assistants because they help developers reduce key-strokes, finish programming tasks quickly, and recall syntax.
We also found the most important reasons why developers do not use these tools are because these tools do not output code that addresses certain functional or non-functional requirements.
arXiv Detail & Related papers (2023-03-30T03:21:53Z) - How Generative AI models such as ChatGPT can be (Mis)Used in SPC
Practice, Education, and Research? An Exploratory Study [2.0841728192954663]
Generative Artificial Intelligence (AI) models have the potential to revolutionize Statistical Process Control (SPC) practice, learning, and research.
These tools are in the early stages of development and can be easily misused or misunderstood.
We explore ChatGPT's ability to provide code, explain basic concepts, and create knowledge related to SPC practice, learning, and research.
arXiv Detail & Related papers (2023-02-17T15:48:37Z) - Interactive Code Generation via Test-Driven User-Intent Formalization [60.90035204567797]
Large language models (LLMs) produce code from informal natural language (NL) intent.
It is hard to define a notion of correctness since natural language can be ambiguous and lacks a formal semantics.
We describe a language-agnostic abstract algorithm and a concrete implementation TiCoder.
arXiv Detail & Related papers (2022-08-11T17:41:08Z) - Competition-Level Code Generation with AlphaCode [74.87216298566942]
We introduce AlphaCode, a system for code generation that can create novel solutions to problems that require deeper reasoning.
In simulated evaluations on recent programming competitions on the Codeforces platform, AlphaCode achieved on average a ranking of top 54.3%.
arXiv Detail & Related papers (2022-02-08T23:16:31Z) - BUSTLE: Bottom-Up Program Synthesis Through Learning-Guided Exploration [72.88493072196094]
We present a new synthesis approach that leverages learning to guide a bottom-up search over programs.
In particular, we train a model to prioritize compositions of intermediate values during search conditioned on a set of input-output examples.
We show that the combination of learning and bottom-up search is remarkably effective, even with simple supervised learning approaches.
arXiv Detail & Related papers (2020-07-28T17:46:18Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.