Execution Guided Line-by-Line Code Generation

• URL: http://arxiv.org/abs/2506.10948v1
• Date: Thu, 12 Jun 2025 17:50:05 GMT
• Title: Execution Guided Line-by-Line Code Generation
• Authors: Boaz Lavon, Shahar Katz, Lior Wolf,
• Abstract summary: We present a novel approach to neural code generation that incorporates real-time execution signals into the language model generation process.<n>Our method, ExecutionGuidedFree Guidance (EGCFG), incorporates execution signals as model generates code.
• Score: 49.1574468325115
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present a novel approach to neural code generation that incorporates real-time execution signals into the language model generation process. While large language models (LLMs) have demonstrated impressive code generation capabilities, they typically do not utilize execution feedback during inference, a critical signal that human programmers regularly leverage. Our method, Execution-Guided Classifier-Free Guidance (EG-CFG), dynamically incorporates execution signals as the model generates code, providing line-by-line feedback that guides the generation process toward executable solutions. EG-CFG employs a multi-stage process: first, we conduct beam search to sample candidate program completions for each line; second, we extract execution signals by executing these candidates against test cases; and finally, we incorporate these signals into the prompt during generation. By maintaining consistent signals across tokens within the same line and refreshing signals at line boundaries, our approach provides coherent guidance while preserving syntactic structure. Moreover, the method naturally supports native parallelism at the task level in which multiple agents operate in parallel, exploring diverse reasoning paths and collectively generating a broad set of candidate solutions. Our experiments across diverse coding tasks demonstrate that EG-CFG significantly improves code generation performance compared to standard approaches, achieving state-of-the-art results across various levels of complexity, from foundational problems to challenging competitive programming tasks. Our code is available at: https://github.com/boazlavon/eg_cfg

Related papers

• HyCodePolicy: Hybrid Language Controllers for Multimodal Monitoring and Decision in Embodied Agents [6.861838493263133]
  HyCodePolicy is a language-based control framework for embodied agents.<n>It integrates code synthesis, geometric grounding, perceptual monitoring, and iterative repair into a closed-loop programming cycle.<n>Our results demonstrate that HyCodePolicy significantly improves the robustness and sample efficiency of robot manipulation policies.
  arXiv  Detail & Related papers  (2025-08-04T17:18:14Z)
• IFEvalCode: Controlled Code Generation [69.28317223249358]
  The paper introduces forward and backward constraints generation to improve the instruction-following capabilities of Code LLMs.<n>The authors present IFEvalCode, a multilingual benchmark comprising 1.6K test samples across seven programming languages.
  arXiv  Detail & Related papers  (2025-07-30T08:08:48Z)
• LSR-MCTS: Alleviating Long Range Dependency in Code Generation [42.10272627826627]
  Large language models (LLMs) have significantly promoted the development of code generation task.<n>We propose the textbfLSR-MCTS algorithm, which leverages MCTS to determine the code line-by-line and select the optimal path.
  arXiv  Detail & Related papers  (2025-04-10T04:03:25Z)
• CODESIM: Multi-Agent Code Generation and Problem Solving through Simulation-Driven Planning and Debugging [3.3856216159724983]
  We introduce CodeSim, a novel multi-agent code generation framework.<n>As human verifies their understanding of any algorithms through visual simulation, CodeSim uniquely features a method of plan verification.<n>Our framework achieves new state-of-the-art results-(HumanEval 95.1%, MBPP 90.7%, APPS 22%, and CodeContests 29.1%)
  arXiv  Detail & Related papers  (2025-02-08T18:43:59Z)
• VisualCoder: Guiding Large Language Models in Code Execution with Fine-grained Multimodal Chain-of-Thought Reasoning [10.70881967278009]
  We introduce VisualCoder, a simple yet effective approach that enhances code reasoning by integrating multimodal Chain-of-Thought snippets (CoT) reasoning with a visual Control Flow Graph (CFG)<n>We address challenges in multimodal CoT integration through a reference mechanism, ensuring consistency between code and its execution path, thereby improving performance in program behavior prediction, error detection, and output generation.
  arXiv  Detail & Related papers  (2024-10-30T19:07:01Z)
• RGD: Multi-LLM Based Agent Debugger via Refinement and Generation Guidance [0.6062751776009752]
  Large Language Models (LLMs) have shown incredible potential in code generation tasks. LLMs can generate code based on task descriptions, but accuracy remains limited. We introduce a novel architecture of LLM-based agents for code generation and automatic debug: Refinement and Guidance debugger (RGD) RGD decomposes the code generation task into multiple steps, ensuring a clearer workflow and enabling iterative code refinement based on self-reflection and feedback.
  arXiv  Detail & Related papers  (2024-10-02T05:07:02Z)
• CodeIP: A Grammar-Guided Multi-Bit Watermark for Large Language Models of Code [56.019447113206006]
  Large Language Models (LLMs) have achieved remarkable progress in code generation.<n>CodeIP is a novel multi-bit watermarking technique that inserts additional information to preserve provenance details.<n>Experiments conducted on a real-world dataset across five programming languages demonstrate the effectiveness of CodeIP.
  arXiv  Detail & Related papers  (2024-04-24T04:25:04Z)
• InterCode: Standardizing and Benchmarking Interactive Coding with Execution Feedback [50.725076393314964]
  We introduce InterCode, a lightweight, flexible, and easy-to-use framework of interactive coding as a standard reinforcement learning environment. Our framework is language and platform agnostic, uses self-contained Docker environments to provide safe and reproducible execution. We demonstrate InterCode's viability as a testbed by evaluating multiple state-of-the-art LLMs configured with different prompting strategies.
  arXiv  Detail & Related papers  (2023-06-26T17:59:50Z)
• SignalGP-Lite: Event Driven Genetic Programming Library for Large-Scale Artificial Life Applications [62.997667081978825]
  Event-driven genetic programming representations have been shown to outperform traditional imperative representations on interaction-intensive problems. Event-driven approach organizes genome content into modules that are triggered in response to environmental signals. SignalGP library caters to traditional program synthesis applications.
  arXiv  Detail & Related papers  (2021-08-01T07:20:49Z)
• Text Generation with Efficient (Soft) Q-Learning [91.47743595382758]
  Reinforcement learning (RL) offers a more flexible solution by allowing users to plug in arbitrary task metrics as reward. We introduce a new RL formulation for text generation from the soft Q-learning perspective. We apply the approach to a wide range of tasks, including learning from noisy/negative examples, adversarial attacks, and prompt generation.
  arXiv  Detail & Related papers  (2021-06-14T18:48:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.