LLM-Guided Compositional Program Synthesis

• URL: http://arxiv.org/abs/2503.15540v1
• Date: Wed, 12 Mar 2025 00:36:43 GMT
• Title: LLM-Guided Compositional Program Synthesis
• Authors: Ruhma Khan, Sumit Gulwani, Vu Le, Arjun Radhakrishna, Ashish Tiwari, Gust Verbruggen,
• Abstract summary: Large language models (LLMs) have the ability to solve PBE tasks by generating code in different target languages, but they can fail unpredictably.<n>We introduce a novel technique that recovers from failure by constructing simpler subtasks for the LLM to solve.
• Score: 16.867355177975387
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Program synthesis from input-output examples, also called programming by example (PBE), has had tremendous impact on automating end-user tasks. Large language models (LLMs) have the ability to solve PBE tasks by generating code in different target languages, but they can fail unpredictably. To recover for failure, most approaches, such as self-reflection, use the LLM to solve the same task, but with a richer context. We introduce a novel technique that recovers from failure by constructing simpler subtasks for the LLM to solve. Our approach performs compositional program synthesis using LLMs, where LLM not only guides the decomposition of the PBE task into subtasks, but also solves the subtasks. We present different strategies for decomposing the original task. We experimentally show that our approach can solve challenging task instances that are not solved by self-reflection alone.

Related papers

• Interactive and Expressive Code-Augmented Planning with Large Language Models [62.799579304821826]
  Large Language Models (LLMs) demonstrate strong abilities in common-sense reasoning and interactive decision-making. Recent techniques have sought to structure LLM outputs using control flow and other code-adjacent techniques to improve planning performance. We propose REPL-Plan, an LLM planning approach that is fully code-expressive and dynamic.
  arXiv  Detail & Related papers  (2024-11-21T04:23:17Z)
• Can LLMs plan paths with extra hints from solvers? [2.874944508343474]
  Large Language Models (LLMs) have shown remarkable capabilities in natural language processing, mathematical problem solving, and tasks related to program synthesis. This paper explores an approach for enhancing LLM performance in solving a classical robotic planning task by integrating solver-generated feedback.
  arXiv  Detail & Related papers  (2024-10-07T14:00:08Z)
• zsLLMCode: An Effective Approach for Code Embedding via LLM with Zero-Shot Learning [6.976968804436321]
  This paper proposes a novel zero-shot approach, zsLLMCode, to generate code embeddings by using large language models (LLMs) and sentence embedding models.<n>The results have demonstrated the effectiveness and superiority of our method over state-of-the-art unsupervised approaches.
  arXiv  Detail & Related papers  (2024-09-23T01:03:15Z)
• Q*: Improving Multi-step Reasoning for LLMs with Deliberative Planning [53.6472920229013]
  Large Language Models (LLMs) have demonstrated impressive capability in many natural language tasks. LLMs are prone to produce errors, hallucinations and inconsistent statements when performing multi-step reasoning. We introduce Q*, a framework for guiding LLMs decoding process with deliberative planning.
  arXiv  Detail & Related papers  (2024-06-20T13:08:09Z)
• Towards Efficient LLM Grounding for Embodied Multi-Agent Collaboration [70.09561665520043]
  We propose a novel framework for multi-agent collaboration that introduces Reinforced Advantage feedback (ReAd) for efficient self-refinement of plans. We provide theoretical analysis by extending advantage-weighted regression in reinforcement learning to multi-agent systems. Experiments on Over-AI and a difficult variant of RoCoBench show that ReAd surpasses baselines in success rate, and also significantly decreases the interaction steps of agents.
  arXiv  Detail & Related papers  (2024-05-23T08:33:19Z)
• Sub-goal Distillation: A Method to Improve Small Language Agents [21.815417165548187]
  Large Language Models (LLMs) have demonstrated significant promise as agents in interactive tasks. We propose a method for transferring the performance of an LLM with billions of parameters to a much smaller language model. In ScienceWorld, a challenging and multi-task interactive text environment, our method surpasses standard imitation learning based solely on elementary actions by 16.7%.
  arXiv  Detail & Related papers  (2024-05-04T20:34:06Z)
• Toward Self-Improvement of LLMs via Imagination, Searching, and Criticizing [56.75702900542643]
  We introduce AlphaLLM for the self-improvements of Large Language Models.<n>It integrates Monte Carlo Tree Search (MCTS) with LLMs to establish a self-improving loop.<n>Our experimental results show that AlphaLLM significantly enhances the performance of LLMs without additional annotations.
  arXiv  Detail & Related papers  (2024-04-18T15:21:34Z)
• Empowering Large Language Models on Robotic Manipulation with Affordance Prompting [23.318449345424725]
  Large language models fail to interact with the physical world by generating control sequences properly. Existing LLM-based approaches circumvent this problem by relying on additional pre-defined skills or pre-trained sub-policies. We propose a framework called LLM+A(ffordance) where the LLM serves as both the sub-task planner and the motion controller.
  arXiv  Detail & Related papers  (2024-04-17T03:06:32Z)
• LLMs Can't Plan, But Can Help Planning in LLM-Modulo Frameworks [18.068035947969044]
  There is considerable confusion about the role of Large Language Models (LLMs) in planning and reasoning tasks. We argue that auto-regressive LLMs cannot, by themselves, do planning or self-verification. We present a vision of bf LLM-Modulo Frameworks that combine the strengths of LLMs with external model-based verifiers.
  arXiv  Detail & Related papers  (2024-02-02T14:43:18Z)
• SatLM: Satisfiability-Aided Language Models Using Declarative Prompting [68.40726892904286]
  We propose a new satisfiability-aided language modeling (SatLM) approach for improving the reasoning capabilities of large language models (LLMs) We use an LLM to generate a declarative task specification rather than an imperative program and leverage an off-the-shelf automated theorem prover to derive the final answer. We evaluate SATLM on 8 different datasets and show that it consistently outperforms program-aided LMs in the imperative paradigm.
  arXiv  Detail & Related papers  (2023-05-16T17:55:51Z)
• Learning to Plan with Natural Language [111.76828049344839]
  Large Language Models (LLMs) have shown remarkable performance in various basic natural language tasks. For completing the complex task, we still need a plan for the task to guide LLMs to generate the specific solutions step by step. We propose the Learning to Plan method, which involves two phases: (1) In the first learning task plan phase, it iteratively updates the task plan with new step-by-step solutions and behavioral instructions, which are obtained by prompting LLMs to derive from training error feedback.
  arXiv  Detail & Related papers  (2023-04-20T17:09:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.