CodePlan: Repository-level Coding using LLMs and Planning
- URL: http://arxiv.org/abs/2309.12499v1
- Date: Thu, 21 Sep 2023 21:45:17 GMT
- Title: CodePlan: Repository-level Coding using LLMs and Planning
- Authors: Ramakrishna Bairi, Atharv Sonwane, Aditya Kanade, Vageesh D C, Arun
Iyer, Suresh Parthasarathy, Sriram Rajamani, B. Ashok, Shashank Shet
- Abstract summary: We frame repository-level coding as a planning problem and present a task-agnostic framework, called CodePlan.
We evaluate the effectiveness of CodePlan on two repository-level tasks: package migration (C#) and temporal code edits (Python)
Our results show that CodePlan has better match with the ground truth compared to baselines.
- Score: 5.987469779811903
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Software engineering activities such as package migration, fixing errors
reports from static analysis or testing, and adding type annotations or other
specifications to a codebase, involve pervasively editing the entire repository
of code. We formulate these activities as repository-level coding tasks.
Recent tools like GitHub Copilot, which are powered by Large Language Models
(LLMs), have succeeded in offering high-quality solutions to localized coding
problems. Repository-level coding tasks are more involved and cannot be solved
directly using LLMs, since code within a repository is inter-dependent and the
entire repository may be too large to fit into the prompt. We frame
repository-level coding as a planning problem and present a task-agnostic
framework, called CodePlan to solve it. CodePlan synthesizes a multi-step chain
of edits (plan), where each step results in a call to an LLM on a code location
with context derived from the entire repository, previous code changes and
task-specific instructions. CodePlan is based on a novel combination of an
incremental dependency analysis, a change may-impact analysis and an adaptive
planning algorithm.
We evaluate the effectiveness of CodePlan on two repository-level tasks:
package migration (C#) and temporal code edits (Python). Each task is evaluated
on multiple code repositories, each of which requires inter-dependent changes
to many files (between 2-97 files). Coding tasks of this level of complexity
have not been automated using LLMs before. Our results show that CodePlan has
better match with the ground truth compared to baselines. CodePlan is able to
get 5/6 repositories to pass the validity checks (e.g., to build without errors
and make correct code edits) whereas the baselines (without planning but with
the same type of contextual information as CodePlan) cannot get any of the
repositories to pass them.
Related papers
- CodeRAG: Supportive Code Retrieval on Bigraph for Real-World Code Generation [69.684886175768]
Large language models (LLMs) have shown promising performance in automated code generation.
In this paper, we propose CodeRAG, a retrieval-augmented code generation framework.
Experiments show that CodeRAG achieves significant improvements compared to no RAG scenarios.
arXiv Detail & Related papers (2025-04-14T09:51:23Z) - MutaGReP: Execution-Free Repository-Grounded Plan Search for Code-Use [92.28400093066212]
MutaGReP is an approach to search for plans that decompose a user request into natural language steps grounded in a large code repository.
Our plans use less than 5% of the 128K context window for GPT-4o but rival the coding performance of GPT-4o with a context window filled with the repo.
arXiv Detail & Related papers (2025-02-21T18:58:17Z) - Interactive and Expressive Code-Augmented Planning with Large Language Models [62.799579304821826]
Large Language Models (LLMs) demonstrate strong abilities in common-sense reasoning and interactive decision-making.
Recent techniques have sought to structure LLM outputs using control flow and other code-adjacent techniques to improve planning performance.
We propose REPL-Plan, an LLM planning approach that is fully code-expressive and dynamic.
arXiv Detail & Related papers (2024-11-21T04:23:17Z) - Unlocking Reasoning Potential in Large Langauge Models by Scaling Code-form Planning [94.76546523689113]
We introduce CodePlan, a framework that generates and follows textcode-form plans -- pseudocode that outlines high-level, structured reasoning processes.
CodePlan effectively captures the rich semantics and control flows inherent to sophisticated reasoning tasks.
It achieves a 25.1% relative improvement compared with directly generating responses.
arXiv Detail & Related papers (2024-09-19T04:13:58Z) - CodexGraph: Bridging Large Language Models and Code Repositories via Code Graph Databases [13.733229886643041]
Large Language Models (LLMs) excel in stand-alone code tasks like HumanEval and MBPP, but struggle with handling entire code repositories.
Similarity-based retrieval often has low recall in complex tasks, while manual tools and APIs are typically task-specific and require expert knowledge.
We introduce CodexGraph, a system that integrates LLM agents with graph database interfaces extracted from code repositories.
arXiv Detail & Related papers (2024-08-07T17:13:59Z) - CodeUpdateArena: Benchmarking Knowledge Editing on API Updates [77.81663273436375]
We present CodeUpdateArena, a benchmark for knowledge editing in the code domain.
An instance in our benchmark consists of a synthetic API function update paired with a program synthesis example.
Our benchmark covers updates of various types to 54 functions from seven diverse Python packages.
arXiv Detail & Related papers (2024-07-08T17:55:04Z) - Hierarchical Context Pruning: Optimizing Real-World Code Completion with Repository-Level Pretrained Code LLMs [24.00351065427465]
We propose a strategy named Hierarchical Context Pruning (HCP) to construct completion prompts with high informational code content.
The HCP models the code repository at the function level, maintaining the topological dependencies between code files while removing a large amount of irrelevant code content.
arXiv Detail & Related papers (2024-06-26T12:26:16Z) - VersiCode: Towards Version-controllable Code Generation [58.82709231906735]
Large Language Models (LLMs) have made tremendous strides in code generation, but existing research fails to account for the dynamic nature of software development.
We propose two novel tasks aimed at bridging this gap: version-specific code completion (VSCC) and version-aware code migration (VACM)
We conduct an extensive evaluation on VersiCode, which reveals that version-controllable code generation is indeed a significant challenge.
arXiv Detail & Related papers (2024-06-11T16:15:06Z) - How to Understand Whole Software Repository? [64.19431011897515]
An excellent understanding of the whole repository will be the critical path to Automatic Software Engineering (ASE)
We develop a novel method named RepoUnderstander by guiding agents to comprehensively understand the whole repositories.
To better utilize the repository-level knowledge, we guide the agents to summarize, analyze, and plan.
arXiv Detail & Related papers (2024-06-03T15:20:06Z) - Iterative Refinement of Project-Level Code Context for Precise Code Generation with Compiler Feedback [29.136378191436396]
We present CoCoGen, a new code generation approach that uses compiler feedback to improve the LLM-generated code.
CoCoGen first leverages static analysis to identify mismatches between the generated code and the project's context.
It then iteratively aligns and fixes the identified errors using information extracted from the code repository.
arXiv Detail & Related papers (2024-03-25T14:07:27Z) - InstructCoder: Instruction Tuning Large Language Models for Code Editing [26.160498475809266]
We explore the use of Large Language Models (LLMs) to edit code based on user instructions.
InstructCoder is the first instruction-tuning dataset designed to adapt LLMs for general-purpose code editing.
Our findings reveal that open-source LLMs fine-tuned on InstructCoder can significantly enhance the accuracy of code edits.
arXiv Detail & Related papers (2023-10-31T10:15:35Z) - RepoCoder: Repository-Level Code Completion Through Iterative Retrieval
and Generation [96.75695811963242]
RepoCoder is a framework to streamline the repository-level code completion process.
It incorporates a similarity-based retriever and a pre-trained code language model.
It consistently outperforms the vanilla retrieval-augmented code completion approach.
arXiv Detail & Related papers (2023-03-22T13:54:46Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.