BAGEL: Bootstrapping Agents by Guiding Exploration with Language

• URL: http://arxiv.org/abs/2403.08140v2
• Date: Sun, 9 Jun 2024 01:12:26 GMT
• Title: BAGEL: Bootstrapping Agents by Guiding Exploration with Language
• Authors: Shikhar Murty, Christopher Manning, Peter Shaw, Mandar Joshi, Kenton Lee,
• Abstract summary: This work presents BAGEL, a method for bootstrapping language model (LM) agents without human supervision. We use BAGEL demonstrations to adapt a zero shot LM agent at test time via in-context learning over retrieved demonstrations. We find improvements of over 2-13% absolute on ToolQA and MiniWob++, with up to 13x reduction in execution failures.
• Score: 19.08719671800276
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Following natural language instructions by executing actions in digital environments (e.g. web-browsers and REST APIs) is a challenging task for language model (LM) agents. Unfortunately, LM agents often fail to generalize to new environments without human demonstrations. This work presents BAGEL, a method for bootstrapping LM agents without human supervision. BAGEL converts a seed set of randomly explored trajectories or synthetic instructions, into demonstrations, via round-trips between two noisy LM components: an LM labeler which converts a trajectory into a synthetic instruction, and a zero-shot LM agent which maps the synthetic instruction into a refined trajectory. By performing these round-trips iteratively, BAGEL quickly converts the initial distribution of trajectories towards those that are well-described by natural language. We use BAGEL demonstrations to adapt a zero shot LM agent at test time via in-context learning over retrieved demonstrations, and find improvements of over 2-13% absolute on ToolQA and MiniWob++, with up to 13x reduction in execution failures.

Related papers

• TasTe: Teaching Large Language Models to Translate through Self-Reflection [82.83958470745381]
  Large language models (LLMs) have exhibited remarkable performance in various natural language processing tasks. We propose the TasTe framework, which stands for translating through self-reflection. The evaluation results in four language directions on the WMT22 benchmark reveal the effectiveness of our approach compared to existing methods.
  arXiv  Detail & Related papers  (2024-06-12T17:21:21Z)
• Frugal LMs Trained to Invoke Symbolic Solvers Achieve Parameter-Efficient Arithmetic Reasoning [36.8749786658624]
  Large Language Models (LLM) exhibit zero-shot mathematical reasoning capacity as a behavior emergent with scale. We show that small LMs can achieve reasonable arithmetic reasoning if arithmetic word problems are posed as a formalize-then-solve task.
  arXiv  Detail & Related papers  (2023-12-09T13:20:49Z)
• FireAct: Toward Language Agent Fine-tuning [63.06306936820456]
  We argue for the overlooked direction of fine-tuning LMs to obtain language agents. Fine-tuning Llama2-7B with 500 agent trajectories generated by GPT-4 leads to a 77% HotpotQA performance increase. We propose FireAct, a novel approach to fine-tuning LMs with trajectories from multiple tasks and prompting methods.
  arXiv  Detail & Related papers  (2023-10-09T17:58:38Z)
• LeTI: Learning to Generate from Textual Interactions [60.425769582343506]
  We explore LMs' potential to learn from textual interactions (LETI) that not only check their correctness with binary labels but also pinpoint and explain errors in their outputs through textual feedback. Our focus is the code generation task, where the model produces code based on natural language instructions. LETI iteratively fine-tunes the model, using the objective LM, on a concatenation of natural language instructions, LM-generated programs, and textual feedback.
  arXiv  Detail & Related papers  (2023-05-17T15:53:31Z)
• Guiding Pretraining in Reinforcement Learning with Large Language Models [133.32146904055233]
  We describe a method that uses background knowledge from text corpora to shape exploration. This method, called ELLM, rewards an agent for achieving goals suggested by a language model. By leveraging large-scale language model pretraining, ELLM guides agents toward human-meaningful and plausibly useful behaviors without requiring a human in the loop.
  arXiv  Detail & Related papers  (2023-02-13T21:16:03Z)
• Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLP [77.817293104436]
  We propose a framework that relies on passing natural language texts in sophisticated pipelines between an LM and an RM. We have written novel DSP programs for answering questions in open-domain, multi-hop, and conversational settings.
  arXiv  Detail & Related papers  (2022-12-28T18:52:44Z)
• Can Large Language Models Truly Understand Prompts? A Case Study with Negated Prompts [19.43042432631113]
  Previous work has shown that there exists a scaling law between the size of Language Models (LMs) and their zero-shot performance on different downstream NLP tasks. In this work, we show that this phenomenon does not hold when evaluating large LMs on tasks with negated prompts, but instead shows an inverse scaling law.
  arXiv  Detail & Related papers  (2022-09-26T14:05:10Z)
• Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents [111.33545170562337]
  We investigate the possibility of grounding high-level tasks, expressed in natural language, to a chosen set of actionable steps. We find that if pre-trained LMs are large enough and prompted appropriately, they can effectively decompose high-level tasks into low-level plans. We propose a procedure that conditions on existing demonstrations and semantically translates the plans to admissible actions.
  arXiv  Detail & Related papers  (2022-01-18T18:59:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.