CoLadder: Supporting Programmers with Hierarchical Code Generation in
Multi-Level Abstraction
- URL: http://arxiv.org/abs/2310.08699v2
- Date: Tue, 26 Dec 2023 14:13:29 GMT
- Title: CoLadder: Supporting Programmers with Hierarchical Code Generation in
Multi-Level Abstraction
- Authors: Ryan Yen, Jiawen Zhu, Sangho Suh, Haijun Xia, Jian Zhao
- Abstract summary: CoLadder is a system that supports programmers by facilitating hierarchical task decomposition, direct code segment manipulation, and result evaluation.
A user study with 12 experienced programmers showed that CoLadder is effective in helping programmers externalize their problem-solving intentions flexibly.
- Score: 16.325032481071997
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Programmers increasingly rely on Large Language Models (LLMs) for code
generation. However, misalignment between programmers' goals and generated code
complicates the code evaluation process and demands frequent switching between
prompt authoring and code evaluation. Yet, current LLM-driven code assistants
lack sufficient scaffolding to help programmers format intentions from their
overarching goals, a crucial step before translating these intentions into
natural language prompts. To address this gap, we adopted an iterative design
process to gain insights into programmers' strategies when using LLMs for
programming. Building on our findings, we created CoLadder, a system that
supports programmers by facilitating hierarchical task decomposition, direct
code segment manipulation, and result evaluation during prompt authoring. A
user study with 12 experienced programmers showed that CoLadder is effective in
helping programmers externalize their problem-solving intentions flexibly,
improving their ability to evaluate and modify code across various abstraction
levels, from goal to final code implementation.
Related papers
- NoviCode: Generating Programs from Natural Language Utterances by Novices [59.71218039095155]
We present NoviCode, a novel NL Programming task which takes as input an API and a natural language description by a novice non-programmer.
We show that NoviCode is indeed a challenging task in the code synthesis domain, and that generating complex code from non-technical instructions goes beyond the current Text-to-Code paradigm.
arXiv Detail & Related papers (2024-07-15T11:26:03Z) - A Study on Developer Behaviors for Validating and Repairing LLM-Generated Code Using Eye Tracking and IDE Actions [13.58143103712]
GitHub Copilot is a large language model (LLM)-powered code generation tool.
This paper investigates how developers validate and repair code generated by Copilot.
Being aware of the code's provenance led to improved performance, increased search efforts, more frequent Copilot usage, and higher cognitive workload.
arXiv Detail & Related papers (2024-05-25T06:20:01Z) - CodeGRAG: Extracting Composed Syntax Graphs for Retrieval Augmented Cross-Lingual Code Generation [60.799992690487336]
We propose Syntax Graph Retrieval Augmented Code Generation (CodeGRAG) to enhance the performance of LLMs in single-round code generation tasks.
CodeGRAG significantly improves the code generation ability of LLMs and can even offer performance gain for cross-lingual code generation.
arXiv Detail & Related papers (2024-05-03T02:48:55Z) - CodeCloak: A Method for Evaluating and Mitigating Code Leakage by LLM Code Assistants [23.462703429753706]
We propose two complementary methods to mitigate the risk of code leakage when using LLM-based code assistants.
The first is a technique for reconstructing a developer's original from code segments sent to the code assistant service.
The second is CodeCloak, a novel deep reinforcement learning agent that manipulates the prompts before sending them to the code assistant service.
arXiv Detail & Related papers (2024-04-13T19:30:58Z) - Comments as Natural Logic Pivots: Improve Code Generation via Comment Perspective [85.48043537327258]
We propose MANGO (comMents As Natural loGic pivOts), including a comment contrastive training strategy and a corresponding logical comment decoding strategy.
Results indicate that MANGO significantly improves the code pass rate based on the strong baselines.
The robustness of the logical comment decoding strategy is notably higher than the Chain-of-thoughts prompting.
arXiv Detail & Related papers (2024-04-11T08:30:46Z) - If LLM Is the Wizard, Then Code Is the Wand: A Survey on How Code
Empowers Large Language Models to Serve as Intelligent Agents [81.60906807941188]
Large language models (LLMs) are trained on a combination of natural language and formal language (code)
Code translates high-level goals into executable steps, featuring standard syntax, logical consistency, abstraction, and modularity.
arXiv Detail & Related papers (2024-01-01T16:51:20Z) - Function-constrained Program Synthesis [12.55507214959886]
Large language models (LLMs) can generate code in real-time by drawing on all code available in a development environment.
Current systems lack effective recovery methods, forcing users to iteratively re-prompt the model with modified prompts until a sufficient solution is reached.
Our method constrains code-generation to an explicit function set and enabling recovery from failed attempts through automatically generated sub-functions.
arXiv Detail & Related papers (2023-11-27T02:55:34Z) - Test-Case-Driven Programming Understanding in Large Language Models for
Better Code Generation [15.166827643436346]
muFiX is a novel prompting technique to improve the code generation performance of large language models (LLMs)
It first exploits test case analysis to obtain specification understanding and enables a self-improvement process.
muFiX further fixes the specification understanding towards the direction reducing the gap between the provided understanding and the actual understanding.
arXiv Detail & Related papers (2023-09-28T02:58:07Z) - When Do Program-of-Thoughts Work for Reasoning? [51.2699797837818]
We propose complexity-impacted reasoning score (CIRS) to measure correlation between code and reasoning abilities.
Specifically, we use the abstract syntax tree to encode the structural information and calculate logical complexity.
Code will be integrated into the EasyInstruct framework at https://github.com/zjunlp/EasyInstruct.
arXiv Detail & Related papers (2023-08-29T17:22:39Z) - Think Outside the Code: Brainstorming Boosts Large Language Models in
Code Generation [9.904734169174356]
In this paper, we introduce Brainstorm framework for code generation.
It leverages a brainstorming step that generates and selects diverse thoughts on the problem.
Brainstorm significantly enhances the ability of LLMs to solve competition-level programming problems.
arXiv Detail & Related papers (2023-05-18T03:32:54Z) - ReACC: A Retrieval-Augmented Code Completion Framework [53.49707123661763]
We propose a retrieval-augmented code completion framework, leveraging both lexical copying and referring to code with similar semantics by retrieval.
We evaluate our approach in the code completion task in Python and Java programming languages, achieving a state-of-the-art performance on CodeXGLUE benchmark.
arXiv Detail & Related papers (2022-03-15T08:25:08Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.