Why Do LLM-based Web Agents Fail? A Hierarchical Planning Perspective

• URL: http://arxiv.org/abs/2603.14248v1
• Date: Sun, 15 Mar 2026 06:52:06 GMT
• Title: Why Do LLM-based Web Agents Fail? A Hierarchical Planning Perspective
• Abstract summary: Large language model (LLM) web agents are increasingly used for web navigation but remain far from human reliability on realistic, long-horizon tasks.<n>We propose a hierarchical planning framework to analyze web agents across three layers (i.e., high-level planning, low-level execution, and replanning)
• Score: 14.967485941664206
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language model (LLM) web agents are increasingly used for web navigation but remain far from human reliability on realistic, long-horizon tasks. Existing evaluations focus primarily on end-to-end success, offering limited insight into where failures arise. We propose a hierarchical planning framework to analyze web agents across three layers (i.e., high-level planning, low-level execution, and replanning), enabling process-based evaluation of reasoning, grounding, and recovery. Our experiments show that structured Planning Domain Definition Language (PDDL) plans produce more concise and goal-directed strategies than natural language (NL) plans, but low-level execution remains the dominant bottleneck. These results indicate that improving perceptual grounding and adaptive control, not only high-level reasoning, is critical for achieving human-level reliability. This hierarchical perspective provides a principled foundation for diagnosing and advancing LLM web agents.

Related papers

• What We Talk About When We Talk About LLM Planning: Evidence for Two Distinct Planning Abilities [43.87329561965196]
  We argue that task-level variation may reflect distinct latent planning competencies rather than differences along a single ability spectrum.<n>We study this question on ACPBench-Hard by evaluating multiple LLM families under varying test-time reasoning budgets.
  arXiv  Detail & Related papers  (2026-07-13T07:47:17Z)
• Does The Way You Plan Matter? An Empirical Study of Planning Representations for LLM Web Agents [43.297647791392144]
  We introduce PlanAhead, a framework that evaluates the impact of plan representation in agent performance.<n>We first categorize WebArena tasks into 3 difficulty levels, enabling consistent difficulty grading without human annotation.<n>We then systematically evaluate 4 different plan representations on the tasks categorized as hard.
  arXiv  Detail & Related papers  (2026-05-28T13:39:49Z)
• METER: Evaluating Multi-Level Contextual Causal Reasoning in Large Language Models [61.33372454250959]
  Contextual causal reasoning is a critical yet challenging capability for Large Language Models.<n>Existing benchmarks fail to ensure context consistency or cover the full causal hierarchy.<n>We pioneer METER to systematically benchmark LLMs across all three levels of the causal ladder.
  arXiv  Detail & Related papers  (2026-04-13T14:07:11Z)
• HiMAC: Hierarchical Macro-Micro Learning for Long-Horizon LLM Agents [19.63866851076813]
  HiMAC is a hierarchical agentic RL framework that decomposes long-horizon decision-making into macro-level planning and micro-level execution.<n>Our results show that introducing structured hierarchy, rather than increasing model scale alone, is a key factor for enabling robust long-horizon agentic intelligence.
  arXiv  Detail & Related papers  (2026-03-01T08:09:03Z)
• Why Reasoning Fails to Plan: A Planning-Centric Analysis of Long-Horizon Decision Making in LLM Agents [42.09897801169138]
  Large language model (LLM)-based agents exhibit strong step-by-step reasoning capabilities over short horizons, yet often fail to sustain coherent behavior over long planning horizons.<n>We argue that step-wise reasoning induces a form of step-wise greedy policy that is adequate for short horizons but fails in long-horizon planning.<n>We introduce FLARE as a minimal instantiation of future-aware planning to enforce explicit lookahead, value propagation, and limited commitment in a single model.
  arXiv  Detail & Related papers  (2026-01-29T20:52:32Z)
• Learning Affordances at Inference-Time for Vision-Language-Action Models [50.93181349331096]
  In robotics, Vision-Language-Action models (VLAs) offer a promising path towards solving complex control tasks.<n>We introduce Learning from Inference-Time Execution (LITEN), which connects a VLA low-level policy to a high-level VLM that conditions on past experiences.<n>Our approach iterates between a reasoning phase that generates and executes plans for the low-level VLA, and an assessment phase that reflects on the resulting execution.
  arXiv  Detail & Related papers  (2025-10-22T16:43:29Z)
• DeepPlanner: Scaling Planning Capability for Deep Research Agents via Advantage Shaping [74.34061104176554]
  We propose DeepPlanner, an end-to-end RL framework that effectively enhances the planning capabilities of deep research agents.<n>Our approach shapes token-level advantage with an entropy-based term to allocate larger updates to high entropy tokens, and selectively upweights sample-level advantages for planning-intensive rollouts.
  arXiv  Detail & Related papers  (2025-10-14T20:47:05Z)
• Emergent Hierarchical Reasoning in LLMs through Reinforcement Learning [56.496001894673235]
  Reinforcement Learning (RL) has proven highly effective at enhancing the complex reasoning abilities of Large Language Models (LLMs)<n>Our analysis reveals that puzzling phenomena like aha moments", length-scaling'' and entropy dynamics are not disparate occurrences but hallmarks of an emergent reasoning hierarchy.
  arXiv  Detail & Related papers  (2025-09-03T18:52:49Z)
• Strict Subgoal Execution: Reliable Long-Horizon Planning in Hierarchical Reinforcement Learning [5.274804664403783]
  Strict Subgoal Execution (SSE) is a graph-based hierarchical RL framework that enforces single-step subgoal reachability.<n>We show that SSE consistently outperforms existing goal-conditioned RL and hierarchical RL approaches in both efficiency and success rate.
  arXiv  Detail & Related papers  (2025-06-26T06:35:42Z)
• PGPO: Enhancing Agent Reasoning via Pseudocode-style Planning Guided Preference Optimization [58.465778756331574]
  We propose a pseudocode-style Planning Guided Preference Optimization method called PGPO for effective agent learning.<n>With two planning-oriented rewards, PGPO further enhances LLM agents' ability to generate high-quality P-code Plans.<n>Experiments show that PGPO achieves superior performance on representative agent benchmarks and outperforms the current leading baselines.
  arXiv  Detail & Related papers  (2025-06-02T09:35:07Z)
• Learning Hierarchical Domain Models Through Environment-Grounded Interaction [14.98687521721637]
  In open-world environments, a single general domain model cannot capture the variety of tasks.<n>Large Language Models (LLMs) can generate such domains, but suffer from high error rates that limit their applicability.<n>We propose LODGE, a framework for autonomous domain learning from LLMs and environment grounding.
  arXiv  Detail & Related papers  (2025-05-15T20:23:21Z)
• Exploring and Benchmarking the Planning Capabilities of Large Language Models [57.23454975238014]
  This work lays the foundations for improving planning capabilities of large language models (LLMs) We construct a comprehensive benchmark suite encompassing both classical planning benchmarks and natural language scenarios. We investigate the use of many-shot in-context learning to enhance LLM planning, exploring the relationship between increased context length and improved planning performance.
  arXiv  Detail & Related papers  (2024-06-18T22:57:06Z)
• LLM-SAP: Large Language Models Situational Awareness Based Planning [0.0]
  We employ a multi-agent reasoning framework to develop a methodology that anticipates and actively mitigates potential risks. Our approach diverges from traditional automata theory by incorporating the complexity of human-centric interactions into the planning process.
  arXiv  Detail & Related papers  (2023-12-26T17:19:09Z)
• Learning adaptive planning representations with natural language guidance [90.24449752926866]
  This paper describes Ada, a framework for automatically constructing task-specific planning representations. Ada interactively learns a library of planner-compatible high-level action abstractions and low-level controllers adapted to a particular domain of planning tasks.
  arXiv  Detail & Related papers  (2023-12-13T23:35:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.