Embodied Web Agents: Bridging Physical-Digital Realms for Integrated Agent Intelligence

• URL: http://arxiv.org/abs/2506.15677v3
• Date: Tue, 29 Jul 2025 22:40:49 GMT
• Title: Embodied Web Agents: Bridging Physical-Digital Realms for Integrated Agent Intelligence
• Authors: Yining Hong, Rui Sun, Bingxuan Li, Xingcheng Yao, Maxine Wu, Alexander Chien, Da Yin, Ying Nian Wu, Zhecan James Wang, Kai-Wei Chang,
• Abstract summary: Embodied Web Agents is a novel paradigm for AI agents that fluidly bridge the embodiment and web-scale reasoning.<n>We release the Embodied Web Agents Benchmark, which encompasses a diverse suite of tasks.<n>Results reveal significant performance gaps between state-of-the-art AI systems and human capabilities.
• Score: 109.32705135051486
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: AI agents today are mostly siloed - they either retrieve and reason over vast amount of digital information and knowledge obtained online; or interact with the physical world through embodied perception, planning and action - but rarely both. This separation limits their ability to solve tasks that require integrated physical and digital intelligence, such as cooking from online recipes, navigating with dynamic map data, or interpreting real-world landmarks using web knowledge. We introduce Embodied Web Agents, a novel paradigm for AI agents that fluidly bridge embodiment and web-scale reasoning. To operationalize this concept, we first develop the Embodied Web Agents task environments, a unified simulation platform that tightly integrates realistic 3D indoor and outdoor environments with functional web interfaces. Building upon this platform, we construct and release the Embodied Web Agents Benchmark, which encompasses a diverse suite of tasks including cooking, navigation, shopping, tourism, and geolocation - all requiring coordinated reasoning across physical and digital realms for systematic assessment of cross-domain intelligence. Experimental results reveal significant performance gaps between state-of-the-art AI systems and human capabilities, establishing both challenges and opportunities at the intersection of embodied cognition and web-scale knowledge access. All datasets, codes and websites are publicly available at our project page https://embodied-web-agent.github.io/.

Related papers

• Agentic Web: Weaving the Next Web with AI Agents [109.13815627467514]
  The emergence of AI agents powered by large language models (LLMs) marks a pivotal shift toward the Agentic Web.<n>In this paradigm, agents interact directly with one another to plan, coordinate, and execute complex tasks on behalf of users.<n>We present a structured framework for understanding and building the Agentic Web.
  arXiv  Detail & Related papers  (2025-07-28T17:58:12Z)
• Embodied AI Agents: Modeling the World [188.85697524284834]
  This paper describes our research on AI agents embodied in visual, virtual or physical forms.<n>We propose that the development of world models is central to reasoning and planning of embodied AI agents.<n>We also propose to learn the mental world model of users to enable better human-agent collaboration.
  arXiv  Detail & Related papers  (2025-06-27T16:05:34Z)
• Reasoning in visual navigation of end-to-end trained agents: a dynamical systems approach [23.52028824411467]
  We present a large-scale experimental study involving numepisodes navigation episodes in a real environment with a physical robot.<n>We analyze the type of reasoning emerging from end-to-end training.<n>We show in a post-hoc analysis that the value function learned by the agent relates to long-term planning.
  arXiv  Detail & Related papers  (2025-03-11T11:16:47Z)
• Proposer-Agent-Evaluator(PAE): Autonomous Skill Discovery For Foundation Model Internet Agents [64.75036903373712]
  Proposer-Agent-Evaluator is a learning system that enables foundation model agents to autonomously discover and practice skills in the wild.<n>At the heart of PAE is a context-aware task proposer that autonomously proposes tasks for the agent to practice with context information.<n>The success evaluation serves as the reward signal for the agent to refine its policies through RL.
  arXiv  Detail & Related papers  (2024-12-17T18:59:50Z)
• EmbodiedCity: A Benchmark Platform for Embodied Agent in Real-world City Environment [38.14321677323052]
  Embodied artificial intelligence emphasizes the role of an agent's body in generating human-like behaviors. In this paper, we construct a benchmark platform for embodied intelligence evaluation in real-world city environments.
  arXiv  Detail & Related papers  (2024-10-12T17:49:26Z)
• MMInA: Benchmarking Multihop Multimodal Internet Agents [36.173995299002]
  We present MMInA, a multihop and multimodal benchmark to evaluate the embodied agents for compositional Internet tasks. Our data includes 1,050 human-written tasks covering various domains such as shopping and travel. Our method significantly improved both the single-hop and multihop web browsing abilities of agents.
  arXiv  Detail & Related papers  (2024-04-15T17:59:50Z)
• V-IRL: Grounding Virtual Intelligence in Real Life [65.87750250364411]
  V-IRL is a platform that enables agents to interact with the real world in a virtual yet realistic environment. Our platform serves as a playground for developing agents that can accomplish various practical tasks.
  arXiv  Detail & Related papers  (2024-02-05T18:59:36Z)
• WebArena: A Realistic Web Environment for Building Autonomous Agents [92.3291458543633]
  We build an environment for language-guided agents that is highly realistic and reproducible. We focus on agents that perform tasks on the web, and create an environment with fully functional websites from four common domains. We release a set of benchmark tasks focusing on evaluating the functional correctness of task completions.
  arXiv  Detail & Related papers  (2023-07-25T22:59:32Z)
• Intelligent Software Web Agents: A Gap Analysis [0.0]
  We examine the status quo in terms of intelligent software web agents, guided by research with respect to requirements and architectural components. We propose a hybrid semantic web agent architecture, discuss the role played by existing semantic web standards, and point to existing work in the broader semantic web community any beyond that could help us to make the semantic web agent vision a reality.
  arXiv  Detail & Related papers  (2021-02-12T16:32:02Z)
• ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulation [75.0278287071591]
  ThreeDWorld (TDW) is a platform for interactive multi-modal physical simulation. TDW enables simulation of high-fidelity sensory data and physical interactions between mobile agents and objects in rich 3D environments. We present initial experiments enabled by TDW in emerging research directions in computer vision, machine learning, and cognitive science.
  arXiv  Detail & Related papers  (2020-07-09T17:33:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.