Region4Web: Rethinking Observation Space Granularity for Web Agents

• URL: http://arxiv.org/abs/2605.07134v1
• Date: Fri, 08 May 2026 02:11:23 GMT
• Title: Region4Web: Rethinking Observation Space Granularity for Web Agents
• Abstract summary: We argue observation should operate at the granularity of functional regions, parts of the page that each serve a distinct purpose.<n>We propose Region4Web, a framework that reorganizes the AXTree into functional regions through hierarchical decomposition and semantic abstraction.<n>We propose PageDigest, a web-specific inference pipeline that delivers this region-level observation to the actor agent as a compact per-page digest that persists across steps.
• Score: 10.39657194669009
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Web agents perceive web pages through an observation space, yet its granularity has remained an underexamined design choice. Existing work treats observation at the same element-level granularity as the action space, leaving the page's functional organization implicit and forcing the agent to infer it from element-level signals at every step. We argue observation should instead operate at the granularity of functional regions, parts of the page that each serve a distinct purpose. We propose Region4Web, a framework that reorganizes the AXTree into functional regions through hierarchical decomposition and semantic abstraction, exposing the page's functional organization as the basis for page state understanding. Moreover, we propose PageDigest, a web-specific inference pipeline that delivers this region-level observation to the actor agent as a compact per-page digest that persists across steps. On the WebArena benchmark, PageDigest substantially reduces observation length while improving overall task success rate across diverse backbone large language models (LLMs) and established agent methods, regardless of backbone capacity. These results show that operating at the granularity of functional regions delivers a more compact and informative basis for the actor agent than element-level processing alone.

Related papers

• Seek to Segment: Active Perception for Panoramic Referring Segmentation [76.39419654494505]
  Existing referring segmentation models passively process static images captured from fixed perspectives.<n>We propose PanoSeeker, a memory-augmented agent for efficient Active Panoramic Referring.<n>Experiments on our newly established APRS benchmark demonstrate that PanoSeeker achieves superior search efficiency and segmentation accuracy.
  arXiv  Detail & Related papers  (2026-07-02T17:56:49Z)
• WebChallenger: A Reliable and Efficient Generalist Web Agent [3.8083338863305882]
  WebChallenger is a web agent framework that addresses each gap through architecture design rather than model scale.<n>Our system achieves 56.3% on WebArena, 48.7% on VisualWebArena, 51.0% on Online-Mind2Web, and 70.9% on WorkArena.
  arXiv  Detail & Related papers  (2026-06-09T04:53:19Z)
• Enhancing Web Agents with a Hierarchical Memory Tree [35.249218954574424]
  Large language model-based web agents have shown strong potential in automating web interactions through advanced reasoning and instruction following.<n>While retrieval-based memory derived from historical trajectories enables these agents to handle complex, long-horizon tasks, current methods struggle to generalize across unseen websites.<n>We identify that this challenge arises from the flat memory structures that entangle high-level task logic with site-specific action details.<n>We propose Hierarchical Memory Tree (HMT), a structured framework designed to explicitly decouple logical planning from action execution.
  arXiv  Detail & Related papers  (2026-03-07T04:07:47Z)
• Thinking on Maps: How Foundation Model Agents Explore, Remember, and Reason Map Environments [10.485672302572368]
  Map environments provide a fundamental medium for representing spatial structure. Understanding how foundation model (FM) agents understand and act in such environments is critical for enabling reliable map-based reasoning and applications.<n>We propose an interactive evaluation framework to analyze how FM agents explore, remember, and reason in symbolic map environments.
  arXiv  Detail & Related papers  (2025-12-30T23:04:29Z)
• Connecting the Dots: Training-Free Visual Grounding via Agentic Reasoning [63.109585527799005]
  GroundingAgent is a visual grounding framework that operates without task-specific fine-tuning.<n>It achieves an average zero-shot grounding accuracy of 65.1 % on widely-used benchmarks.<n>It also offers strong interpretability, transparently illustrating each reasoning step.
  arXiv  Detail & Related papers  (2025-11-24T03:11:08Z)
• From Text to Space: Mapping Abstract Spatial Models in LLMs during a Grid-World Navigation Task [0.0]
  We investigate the influence of different text-based spatial representations on large language models (LLMs) performance and internal activations in a grid-world navigation task.<n>Our experiments reveal that cartesian representations of space consistently yield higher success rates and path efficiency, with performance scaling markedly with model size.<n>This work advances our understanding of how LLMs process spatial information and provides valuable insights for developing more interpretable and robust agentic AI systems.
  arXiv  Detail & Related papers  (2025-02-23T19:09:01Z)
• Scalable spectral representations for multi-agent reinforcement learning in network MDPs [13.782868855372774]
  A popular model for multi-agent control, Network Markov Decision Processes (MDPs) pose a significant challenge to efficient learning. We first derive scalable spectral local representations for network MDPs, which induces a network linear subspace for the local $Q$-function of each agent. We design a scalable algorithmic framework for continuous state-action network MDPs, and provide end-to-end guarantees for the convergence of our algorithm.
  arXiv  Detail & Related papers  (2024-10-22T17:45:45Z)
• AgentOccam: A Simple Yet Strong Baseline for LLM-Based Web Agents [52.13695464678006]
  This study enhances an LLM-based web agent by simply refining its observation and action space.<n>AgentOccam surpasses the previous state-of-the-art and concurrent work by 9.8 (+29.4%) and 5.9 (+15.8%) absolute points respectively.
  arXiv  Detail & Related papers  (2024-10-17T17:50:38Z)
• Framework-agnostic Semantically-aware Global Reasoning for Segmentation [29.69187816377079]
  We propose a component that learns to project image features into latent representations and reason between them. Our design encourages the latent regions to represent semantic concepts by ensuring that the activated regions are spatially disjoint. Our latent tokens are semantically interpretable and diverse and provide a rich set of features that can be transferred to downstream tasks.
  arXiv  Detail & Related papers  (2022-12-06T21:42:05Z)
• Part-aware Prototypical Graph Network for One-shot Skeleton-based Action Recognition [57.86960990337986]
  One-shot skeleton-based action recognition poses unique challenges in learning transferable representation from base classes to novel classes. We propose a part-aware prototypical representation for one-shot skeleton-based action recognition. We demonstrate the effectiveness of our method on two public skeleton-based action recognition datasets.
  arXiv  Detail & Related papers  (2022-08-19T04:54:56Z)
• TIE: Topological Information Enhanced Structural Reading Comprehension on Web Pages [31.291568831285442]
  We propose a Topological Information Enhanced model (TIE) to transform the token-level task into a tag-level task. TIE integrates Graph Attention Network (GAT) and Pre-trained Language Model (PLM) to leverage the information. Experimental results demonstrate that our model outperforms strong baselines and achieves both logical structures and spatial structures.
  arXiv  Detail & Related papers  (2022-05-13T03:21:09Z)
• Global Context-Aware Progressive Aggregation Network for Salient Object Detection [117.943116761278]
  We propose a novel network named GCPANet to integrate low-level appearance features, high-level semantic features, and global context features. We show that the proposed approach outperforms the state-of-the-art methods both quantitatively and qualitatively.
  arXiv  Detail & Related papers  (2020-03-02T04:26:10Z)
• SceneEncoder: Scene-Aware Semantic Segmentation of Point Clouds with A Learnable Scene Descriptor [51.298760338410624]
  We propose a SceneEncoder module to impose a scene-aware guidance to enhance the effect of global information. The module predicts a scene descriptor, which learns to represent the categories of objects existing in the scene. We also design a region similarity loss to propagate distinguishing features to their own neighboring points with the same label.
  arXiv  Detail & Related papers  (2020-01-24T16:53:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.