Does The Way You Plan Matter? An Empirical Study of Planning Representations for LLM Web Agents

• URL: http://arxiv.org/abs/2605.29927v1
• Date: Thu, 28 May 2026 13:39:49 GMT
• Title: Does The Way You Plan Matter? An Empirical Study of Planning Representations for LLM Web Agents
• Abstract summary: We introduce PlanAhead, a framework that evaluates the impact of plan representation in agent performance.<n>We first categorize WebArena tasks into 3 difficulty levels, enabling consistent difficulty grading without human annotation.<n>We then systematically evaluate 4 different plan representations on the tasks categorized as hard.
• Score: 43.297647791392144
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Despite recent advances, LLM-based web agents still struggle with limited exploration, omission of critical steps, and sensitivity to task constraints. Prior work suggests that many of these failures stem from weaknesses in planning, yet the impact of alternative natural language plan representation remains unexplored. To address this, we introduce PlanAhead, a static planner-executor framework that evaluates the impact of plan representation in agent performance. We first automatically categorize WebArena tasks into 3 difficulty levels, enabling consistent difficulty grading without human annotation. Then we systematically evaluate 4 different plan representations on the tasks categorized as hard: sequential subgoals, narrative, pseudocode, and checklist; across different families of multimodal LLM powered agents (OpenAI, Alibaba, and Google). To account for stochastic variability, we introduce two novel evaluation metrics: Achievement Rate (AR) and Solved-Task Consistency (STC). Our results show that both, the plan formulation and the underlying LLM generating the plan, significantly influence web-agent robustness and task success.

Related papers

• From Plan to Action: How Well Do Agents Follow the Plan? [6.54753809716901]
  This paper presents the first systematic analysis of plan compliance in programming agents.<n>We examined 16,991 trajectories from SWE-agents across four LLMs on SWE-bench Verified and SWEbench Pro under eight plan variations.<n>A subpar plan hurts performance even more than no plan at all.
  arXiv  Detail & Related papers  (2026-04-13T23:54:55Z)
• Plan-and-Act: Improving Planning of Agents for Long-Horizon Tasks [36.63527489464188]
  Plan-and-Act is a framework that incorporates explicit planning into large language models (LLMs)<n>Plan-and-Act consists of a Planner model which generates structured, high-level plans to achieve user goals, and an Executor model that translates these plans into environment-specific actions.<n>We present a state-of-the-art 57.58% success rate on the WebArena-Lite benchmark as well as a text-only state-of-the-art 81.36% success rate on WebVoyager.
  arXiv  Detail & Related papers  (2025-03-12T17:40:52Z)
• PlanGenLLMs: A Modern Survey of LLM Planning Capabilities [12.322175348741435]
  LLMs have immense potential for generating plans, transforming an initial world state into a desired goal state.<n>Many of these systems are tailored to specific problems, making it challenging to compare them or determine the best approach for new tasks.<n>Our survey aims to offer a comprehensive overview of current LLM planners to fill this gap.<n>It builds on foundational work by Kartam and Wilkins (1990) and examines six key performance criteria: completeness, executability, optimality, representation, generalization, and efficiency.
  arXiv  Detail & Related papers  (2025-02-16T17:54:57Z)
• KnowAgent: Knowledge-Augmented Planning for LLM-Based Agents [52.34892973785117]
  Large Language Models (LLMs) have demonstrated great potential in complex reasoning tasks, yet they fall short when tackling more sophisticated challenges.<n>This inadequacy primarily stems from the lack of built-in action knowledge in language agents.<n>We introduce KnowAgent, a novel approach designed to enhance the planning capabilities of LLMs by incorporating explicit action knowledge.
  arXiv  Detail & Related papers  (2024-03-05T16:39:12Z)
• ADaPT: As-Needed Decomposition and Planning with Language Models [131.063805299796]
  We introduce As-Needed Decomposition and Planning for complex Tasks (ADaPT) ADaPT explicitly plans and decomposes complex sub-tasks as-needed, when the Large Language Models is unable to execute them. Our results demonstrate that ADaPT substantially outperforms established strong baselines.
  arXiv  Detail & Related papers  (2023-11-08T17:59:15Z)
• AdaPlanner: Adaptive Planning from Feedback with Language Models [56.367020818139665]
  Large language models (LLMs) have recently demonstrated the potential in acting as autonomous agents for sequential decision-making tasks. We propose a closed-loop approach, AdaPlanner, which allows the LLM agent to refine its self-generated plan adaptively in response to environmental feedback. To mitigate hallucination, we develop a code-style LLM prompt structure that facilitates plan generation across a variety of tasks, environments, and agent capabilities.
  arXiv  Detail & Related papers  (2023-05-26T05:52:27Z)
• Plan, Eliminate, and Track -- Language Models are Good Teachers for Embodied Agents [99.17668730578586]
  Pre-trained large language models (LLMs) capture procedural knowledge about the world. Plan, Eliminate, and Track (PET) framework translates a task description into a list of high-level sub-tasks. PET framework leads to a significant 15% improvement over SOTA for generalization to human goal specifications.
  arXiv  Detail & Related papers  (2023-05-03T20:11:22Z)
• Learning to Plan with Natural Language [111.76828049344839]
  Large Language Models (LLMs) have shown remarkable performance in various basic natural language tasks. For completing the complex task, we still need a plan for the task to guide LLMs to generate the specific solutions step by step. We propose the Learning to Plan method, which involves two phases: (1) In the first learning task plan phase, it iteratively updates the task plan with new step-by-step solutions and behavioral instructions, which are obtained by prompting LLMs to derive from training error feedback.
  arXiv  Detail & Related papers  (2023-04-20T17:09:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.