Is Your LLM Secretly a World Model of the Internet? Model-Based Planning for Web Agents

• URL: http://arxiv.org/abs/2411.06559v1
• Date: Sun, 10 Nov 2024 18:50:51 GMT
• Title: Is Your LLM Secretly a World Model of the Internet? Model-Based Planning for Web Agents
• Authors: Yu Gu, Boyuan Zheng, Boyu Gou, Kai Zhang, Cheng Chang, Sanjari Srivastava, Yanan Xie, Peng Qi, Huan Sun, Yu Su,
• Abstract summary: We introduce a novel paradigm that augments language agents with model-based planning. Our method, WebDreamer, builds on the key insight that LLMs inherently encode comprehensive knowledge about website structures and functionalities.
• Score: 23.1522773245956
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Language agents have demonstrated promising capabilities in automating web-based tasks, though their current reactive approaches still underperform largely compared to humans. While incorporating advanced planning algorithms, particularly tree search methods, could enhance these agents' performance, implementing tree search directly on live websites poses significant safety risks and practical constraints due to irreversible actions such as confirming a purchase. In this paper, we introduce a novel paradigm that augments language agents with model-based planning, pioneering the innovative use of large language models (LLMs) as world models in complex web environments. Our method, WebDreamer, builds on the key insight that LLMs inherently encode comprehensive knowledge about website structures and functionalities. Specifically, WebDreamer uses LLMs to simulate outcomes for each candidate action (e.g., "what would happen if I click this button?") using natural language descriptions, and then evaluates these imagined outcomes to determine the optimal action at each step. Empirical results on two representative web agent benchmarks with online interaction -- VisualWebArena and Mind2Web-live -- demonstrate that WebDreamer achieves substantial improvements over reactive baselines. By establishing the viability of LLMs as world models in web environments, this work lays the groundwork for a paradigm shift in automated web interaction. More broadly, our findings open exciting new avenues for future research into 1) optimizing LLMs specifically for world modeling in complex, dynamic environments, and 2) model-based speculative planning for language agents.

Related papers

• AgentOccam: A Simple Yet Strong Baseline for LLM-Based Web Agents [52.13695464678006]
  This study enhances an LLM-based web agent by simply refining its observation and action space. AgentOccam surpasses the previous state-of-the-art and concurrent work by 9.8 (+29.4%) and 5.9 (+15.8%) absolute points respectively.
  arXiv  Detail & Related papers  (2024-10-17T17:50:38Z)
• Web Agents with World Models: Learning and Leveraging Environment Dynamics in Web Navigation [25.26545170310844]
  We present a World-model-augmented (WMA) web agent, which simulates the outcomes of its actions for better decision-making. Experiments on WebArena and Mind2Web show that our world models improve agents' policy selection without training.
  arXiv  Detail & Related papers  (2024-10-17T05:37:00Z)
• Making Large Language Models into World Models with Precondition and Effect Knowledge [1.8561812622368763]
  We show that Large Language Models (LLMs) can be induced to perform two critical world model functions. We validate that the precondition and effect knowledge generated by our models aligns with human understanding of world dynamics.
  arXiv  Detail & Related papers  (2024-09-18T19:28:04Z)
• AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks? [50.36826943689364]
  We study whether language agents can perform realistic and time-consuming tasks on the web. We introduce AssistantBench, a new benchmark consisting of 214 realistic tasks that can be automatically evaluated. We find that AssistantBench exposes the limitations of current systems, including language models and retrieval-augmented language models.
  arXiv  Detail & Related papers  (2024-07-22T15:18:45Z)
• Tree Search for Language Model Agents [69.43007235771383]
  We propose an inference-time search algorithm for LM agents to perform exploration and multi-step planning in interactive web environments. Our approach is a form of best-first tree search that operates within the actual environment space. It is the first tree search algorithm for LM agents that shows effectiveness on realistic web tasks.
  arXiv  Detail & Related papers  (2024-07-01T17:07:55Z)
• LangSuitE: Planning, Controlling and Interacting with Large Language Models in Embodied Text Environments [70.91258869156353]
  We introduce LangSuitE, a versatile and simulation-free testbed featuring 6 representative embodied tasks in textual embodied worlds. Compared with previous LLM-based testbeds, LangSuitE offers adaptability to diverse environments without multiple simulation engines. We devise a novel chain-of-thought (CoT) schema, EmMem, which summarizes embodied states w.r.t. history information.
  arXiv  Detail & Related papers  (2024-06-24T03:36:29Z)
• Large Language Models Can Self-Improve At Web Agent Tasks [37.17001438055515]
  Large language models (LLMs) have recently demonstrated some capability to navigate novel environments as agents in a zero-shot or few-shot fashion. We explore the extent to which LLMs can self-improve their performance as agents in long-horizon tasks in a complex environment using the WebArena benchmark. We achieve a 31% improvement in task completion rate over the base model on the WebArena benchmark through a self-improvement procedure.
  arXiv  Detail & Related papers  (2024-05-30T17:52:36Z)
• On the Multi-turn Instruction Following for Conversational Web Agents [83.51251174629084]
  We introduce a new task of Conversational Web Navigation, which necessitates sophisticated interactions that span multiple turns with both the users and the environment. We propose a novel framework, named self-reflective memory-augmented planning (Self-MAP), which employs memory utilization and self-reflection techniques.
  arXiv  Detail & Related papers  (2024-02-23T02:18:12Z)
• WorldCoder, a Model-Based LLM Agent: Building World Models by Writing Code and Interacting with the Environment [11.81398773711566]
  We give a model-based agent that builds a Python program representing its knowledge of the world based on its interactions with the environment. We study our agent on gridworlds, and on task planning, finding our approach is more sample-efficient compared to deep RL, more compute-efficient compared to ReAct-style agents, and that it can transfer its knowledge across environments by editing its code.
  arXiv  Detail & Related papers  (2024-02-19T16:39:18Z)
• AllTogether: Investigating the Efficacy of Spliced Prompt for Web Navigation using Large Language Models [2.234037966956278]
  We introduce AllTogether, a standardized prompt template that enhances task context representation. We evaluate the efficacy of this approach through prompt learning and instruction finetuning based on open-source Llama-2 and API-accessible GPT models.
  arXiv  Detail & Related papers  (2023-10-20T11:10:14Z)
• Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Models [31.509994889286183]
  We introduce Language Agent Tree Search (LATS) -- the first general framework that synergizes the capabilities of language models (LMs) in reasoning, acting, and planning. A key feature of our approach is the incorporation of an environment for external feedback, which offers a more deliberate and adaptive problem-solving mechanism. LATS achieves state-of-the-art pass@1 accuracy (92.7%) for programming on HumanEval with GPT-4 and demonstrates gradient-free performance (average score of 75.9) comparable to gradient-based fine-tuning for web navigation on WebShop with GPT
  arXiv  Detail & Related papers  (2023-10-06T17:55:11Z)
• LASER: LLM Agent with State-Space Exploration for Web Navigation [57.802977310392755]
  Large language models (LLMs) have been successfully adapted for interactive decision-making tasks like web navigation. Previous methods implicitly assume a forward-only execution mode for the model, where they only provide oracle trajectories as in-context examples. We propose to model the interactive task as state space exploration, where the LLM agent transitions among a pre-defined set of states by performing actions to complete the task.
  arXiv  Detail & Related papers  (2023-09-15T05:44:08Z)
• A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesis [69.15016747150868]
  We introduce WebAgent, an agent that learns from self-experience to complete tasks on real websites. WebAgent plans ahead by decomposing instructions into canonical sub-instructions, summarizes long HTML documents into task-relevant snippets, and acts on websites. We empirically demonstrate that our modular recipe improves the success on real websites by over 50%, and that HTML-T5 is the best model to solve various HTML understanding tasks.
  arXiv  Detail & Related papers  (2023-07-24T14:56:30Z)
• Grounded Decoding: Guiding Text Generation with Grounded Models for Embodied Agents [111.15288256221764]
  Grounded-decoding project aims to solve complex, long-horizon tasks in a robotic setting by leveraging the knowledge of both models. We frame this as a problem similar to probabilistic filtering: decode a sequence that both has high probability under the language model and high probability under a set of grounded model objectives. We demonstrate how such grounded models can be obtained across three simulation and real-world domains, and that the proposed decoding strategy is able to solve complex, long-horizon tasks in a robotic setting by leveraging the knowledge of both models.
  arXiv  Detail & Related papers  (2023-03-01T22:58:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.