LabUtopia: High-Fidelity Simulation and Hierarchical Benchmark for Scientific Embodied Agents

• URL: http://arxiv.org/abs/2505.22634v1
• Date: Wed, 28 May 2025 17:50:53 GMT
• Title: LabUtopia: High-Fidelity Simulation and Hierarchical Benchmark for Scientific Embodied Agents
• Authors: Rui Li, Zixuan Hu, Wenxi Qu, Jinouwen Zhang, Zhenfei Yin, Sha Zhang, Xuantuo Huang, Hanqing Wang, Tai Wang, Jiangmiao Pang, Wanli Ouyang, Lei Bai, Wangmeng Zuo, Ling-Yu Duan, Dongzhan Zhou, Shixiang Tang,
• Abstract summary: LabUtopia is a comprehensive simulation and benchmarking suite designed to facilitate the development of generalizable, reasoning-capable embodied agents.<n>It supports 30 distinct tasks and includes more than 200 scene and instrument assets.<n>We demonstrate that LabUtopia offers a powerful platform for advancing the integration of perception, planning, and control in scientific-purpose agents.
• Score: 103.65422553044816
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Scientific embodied agents play a crucial role in modern laboratories by automating complex experimental workflows. Compared to typical household environments, laboratory settings impose significantly higher demands on perception of physical-chemical transformations and long-horizon planning, making them an ideal testbed for advancing embodied intelligence. However, its development has been long hampered by the lack of suitable simulator and benchmarks. In this paper, we address this gap by introducing LabUtopia, a comprehensive simulation and benchmarking suite designed to facilitate the development of generalizable, reasoning-capable embodied agents in laboratory settings. Specifically, it integrates i) LabSim, a high-fidelity simulator supporting multi-physics and chemically meaningful interactions; ii) LabScene, a scalable procedural generator for diverse scientific scenes; and iii) LabBench, a hierarchical benchmark spanning five levels of complexity from atomic actions to long-horizon mobile manipulation. LabUtopia supports 30 distinct tasks and includes more than 200 scene and instrument assets, enabling large-scale training and principled evaluation in high-complexity environments. We demonstrate that LabUtopia offers a powerful platform for advancing the integration of perception, planning, and control in scientific-purpose agents and provides a rigorous testbed for exploring the practical capabilities and generalization limits of embodied intelligence in future research.

Related papers

• An AI-native experimental laboratory for autonomous biomolecular engineering [12.382004681010915]
  We present an AI-native autonomous laboratory, targeting highly complex scientific experiments for applications like autonomous biomolecular engineering.<n>This system autonomously manages instrumentation, formulates experiment-specific procedures and optimizations, and concurrently serves multiple user requests.<n>It also enables applications in fields such as disease diagnostics, drug development, and information storage.
  arXiv  Detail & Related papers  (2025-07-03T07:21:19Z)
• BioMARS: A Multi-Agent Robotic System for Autonomous Biological Experiments [8.317138109309967]
  Large language models (LLMs) and vision-language models (VLMs) have the potential to transform biological research by enabling autonomous experimentation.<n>Here we introduce BioMARS, an intelligent platform that integrates LLMs, VLMs, and modular robotics to autonomously design, plan, and execute biological experiments.<n>A web interface enables real-time human-AI collaboration, while a modular backend allows scalable integration with laboratory hardware.
  arXiv  Detail & Related papers  (2025-07-02T08:47:02Z)
• Position: Intelligent Science Laboratory Requires the Integration of Cognitive and Embodied AI [98.19195693735487]
  We propose the paradigm of Intelligent Science Laboratories (ISLs)<n>ISLs are a multi-layered, closed-loop framework that deeply integrates cognitive and embodied intelligence.<n>We argue that such systems are essential for overcoming the current limitations of scientific discovery.
  arXiv  Detail & Related papers  (2025-06-24T13:31:44Z)
• Perspective on Utilizing Foundation Models for Laboratory Automation in Materials Research [6.793869699081147]
  This review explores the potential of foundation models to advance laboratory automation in the materials and chemical sciences.<n>It emphasizes the dual roles of these models: cognitive functions for experimental planning and data analysis, and physical functions for hardware operations.<n>Recent advancements have demonstrated the feasibility of using large language models (LLMs) and multimodal robotic systems to handle complex and dynamic laboratory tasks.
  arXiv  Detail & Related papers  (2025-06-14T02:22:28Z)
• ScienceBoard: Evaluating Multimodal Autonomous Agents in Realistic Scientific Workflows [82.07367406991678]
  Large Language Models (LLMs) have extended their impact beyond Natural Language Processing.<n>Among these, computer-using agents are capable of interacting with operating systems as humans do.<n>We introduce ScienceBoard, which encompasses a realistic, multi-domain environment featuring dynamic and visually rich scientific software.
  arXiv  Detail & Related papers  (2025-05-26T12:27:27Z)
• Autonomous Microscopy Experiments through Large Language Model Agents [4.241267255764773]
  Large language models (LLMs) have accelerated the development of self-driving laboratories (SDLs) for materials research.<n>Here, we introduce AILA (Artificially Intelligent Lab Assistant), a framework that automates atomic force microscopy (AFM) through LLM-driven agents.<n>Our systematic assessment shows that state-of-the-art language models struggle even with basic tasks such as documentation retrieval.
  arXiv  Detail & Related papers  (2024-12-18T09:35:28Z)
• Agents for self-driving laboratories applied to quantum computing [2.840384720502993]
  This paper introduces the k-agents framework, designed to support experimentalists in organizing laboratory knowledge and automating experiments with agents.<n>Our framework employs large language model-based agents to encapsulate laboratory knowledge including available laboratory operations and methods for analyzing experiment results.<n>To automate experiments, we introduce execution agents that break multi-step experimental procedures into state machines, interact with other agents to execute each step and analyze the experiment results.
  arXiv  Detail & Related papers  (2024-12-10T23:30:44Z)
• AlabOS: A Python-based Reconfigurable Workflow Management Framework for Autonomous Laboratories [3.8330070166920556]
  We introduce AlabOS, a general-purpose software framework for orchestrating experiments and managing resources. AlabOS features a reconfigurable experiment workflow model and a resource reservation mechanism, enabling the simultaneous execution of varied tasks. We demonstrate the implementation of AlabOS in a prototype autonomous materials laboratory, A-Lab, with around 3,500 samples synthesized over 1.5 years.
  arXiv  Detail & Related papers  (2024-05-22T18:59:39Z)
• MLXP: A Framework for Conducting Replicable Experiments in Python [63.37350735954699]
  We propose MLXP, an open-source, simple, and lightweight experiment management tool based on Python. It streamlines the experimental process with minimal overhead while ensuring a high level of practitioner overhead.
  arXiv  Detail & Related papers  (2024-02-21T14:22:20Z)
• Chemist-X: Large Language Model-empowered Agent for Reaction Condition Recommendation in Chemical Synthesis [55.30328162764292]
  Chemist-X is a comprehensive AI agent that automates the reaction condition optimization (RCO) task in chemical synthesis.<n>The agent uses retrieval-augmented generation (RAG) technology and AI-controlled wet-lab experiment executions.<n>Results of our automatic wet-lab experiments, achieved by fully LLM-supervised end-to-end operation with no human in the lope, prove Chemist-X's ability in self-driving laboratories.
  arXiv  Detail & Related papers  (2023-11-16T01:21:33Z)
• Octopus: Embodied Vision-Language Programmer from Environmental Feedback [58.04529328728999]
  Embodied vision-language models (VLMs) have achieved substantial progress in multimodal perception and reasoning. To bridge this gap, we introduce Octopus, an embodied vision-language programmer that uses executable code generation as a medium to connect planning and manipulation. Octopus is designed to 1) proficiently comprehend an agent's visual and textual task objectives, 2) formulate intricate action sequences, and 3) generate executable code.
  arXiv  Detail & Related papers  (2023-10-12T17:59:58Z)
• An in-depth experimental study of sensor usage and visual reasoning of robots navigating in real environments [20.105395754497202]
  We study the performance and reasoning capacities of real physical agents, trained in simulation and deployed to two different physical environments. We show, that for the PointGoal task, an agent pre-trained on wide variety of tasks and fine-tuned on a simulated version of the target environment can reach competitive performance without modelling any sim2real transfer.
  arXiv  Detail & Related papers  (2021-11-29T16:27:29Z)
• BEHAVIOR: Benchmark for Everyday Household Activities in Virtual, Interactive, and Ecological Environments [70.18430114842094]
  We introduce BEHAVIOR, a benchmark for embodied AI with 100 activities in simulation. These activities are designed to be realistic, diverse, and complex. We include 500 human demonstrations in virtual reality (VR) to serve as the human ground truth.
  arXiv  Detail & Related papers  (2021-08-06T23:36:23Z)
• Empirica: a virtual lab for high-throughput macro-level experiments [4.077787659104315]
  Empirica is a modular virtual lab that offers a solution to the usability-functionality trade-off. Empirica's architecture is designed to allow for parameterizable experimental designs, reusable protocols, and rapid development.
  arXiv  Detail & Related papers  (2020-06-19T21:28:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.