Fugu-MT: arxivの論文翻訳
Daily Report / 2026-10-26

Daily Paper Briefing

2026-10-26 1 Paper

EmbodiedLab: Benchmarking Embodied Agents That Act to Know

Authors Hongcheng Gao, Zelin Zheng, Alex Zhou, Jay Zhu, Shijia Ge, Yazhe Wang, Xiping Li, Jianshu Zeng, Di Wu, Xuan Shangguan, Yexing Wan, Lingyu He, Yongli Wang, Xiaohan Yu, Zhiqi Jia, Yue Liu, Bryan Hooi, Yinpeng Dong, Wentao Zhang, Sihang Wu, Xiao He
Affiliations HexaFuture Inc. / National University of Singapore / Tsinghua University / Peking University
Categories Evaluation / Agent Benchmarking / Evaluating embodied agent capabilities, Application / Embodied Robotics / Agents interacting with real-world tasks, Method / Perception and Planning / Native observation and control assessment
License CC BY 4.0

Abstract Overview

EmbodiedLab is a benchmark for embodied agents that must physically interact with a system in order to discover required information rather than simply driving the world toward a predefined target state. Across 30 quantitative science experiments spanning five disciplines and 14 apparatus platforms in a simulated laboratory, agents control a robot arm, read instruments through cameras, and devise their own procedures. Submitted results are scored on numerical accuracy against hidden resampled parameters, whether cited evidence was actually recorded during the episode, and whether the bench is restored to the required state. The paper evaluates six frontier foundation models within their developers' coding-agent frameworks across nested interface levels providing varying degrees of perception, planning, and manipulation assistance.

Novelty

The paper formalizes the paradigm of "acting to know," where task-relevant information must be actively discovered through physical experimentation and instrument reading rather than symbolic queries or standard goal-reaching manipulation. EmbodiedLab tightly couples physical actions, visual instrument observation, logged evidence verification, and bench-cleanup requirements into a unified, solvable evaluation suite.

Results

Under native observation and control (T0), even the strongest evaluated agents achieved success rates below 50%, with Claude Opus 5.5 reaching 43.3% and GPT-6 Astra reaching 40.0%. Adding privileged perception and motion planning (T1) improved success rates by only 1.7 to 10.0 percentage points, whereas providing distilled manipulation skills (T2) boosted weaker agents by 10.0 to 26.7 percentage points. Error attribution revealed that 75% to 100% of failed runs stem from physical interaction, observation, and bench restoration rather than analytical reasoning over acquired data.

Key Points

  1. EmbodiedLab introduces 30 verified physics-based laboratory experiments requiring robotic agents to discover hidden parameters through autonomous physical interaction and camera-based instrument reading.
  2. Evaluation evaluates not only numerical correctness but also verifies that cited data was generated during the episode and that the apparatus is safely restored to its required bench state.
  3. Experimental results show that reliable physical interaction and apparatus handling represent the primary bottleneck for frontier coding agents, outweighing high-level scientific reasoning limitations.

References

GitHub Project
This page was created using generative AI such as GPT-5, Claude Opus 4, Gemini 3, Gemini 3.1 Flash Image, and their higher-end successor versions. No guarantee can be made regarding its contents.