AgenticLab: A Real-World Robot Agent Platform that Can See, Think, and Act

• URL: http://arxiv.org/abs/2602.01662v3
• Date: Mon, 09 Mar 2026 04:27:30 GMT
• Title: AgenticLab: A Real-World Robot Agent Platform that Can See, Think, and Act
• Abstract summary: We present AgenticLab, a model-agnostic robot agent platform and benchmark for open-world manipulation.<n>We benchmark state-of-the-art VLM-based agents on real-robot tasks in unstructured environments.<n>Our benchmark reveals several failure modes that offline vision-language tests fail to capture.
• Score: 27.922630781100864
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent advances in large vision-language models (VLMs) have demonstrated generalizable open-vocabulary perception and reasoning, yet their real-robot manipulation capability remains unclear for long-horizon, closed-loop execution in unstructured, in-the-wild environments. Prior VLM-based manipulation pipelines are difficult to compare across different research groups' setups, and many evaluations rely on simulation, privileged state, or specially designed setups. We present AgenticLab, a model-agnostic robot agent platform and benchmark for open-world manipulation. AgenticLab provides a closed-loop agent pipeline for perception, task decomposition, online verification, and replanning. Using AgenticLab, we benchmark state-of-the-art VLM-based agents on real-robot tasks in unstructured environments. Our benchmark reveals several failure modes that offline vision-language tests (e.g., VQA and static image understanding) fail to capture, including breakdowns in multi-step grounding consistency, object grounding under occlusion and scene changes, and insufficient spatial reasoning for reliable manipulation. We will release the full hardware and software stack to support reproducible evaluation and accelerate research on general-purpose robot agents.

Related papers

• DuMateBench: Evaluating Autonomous Agents in Complex Real-World Workflows [69.12339622053588]
  We introduce DuMateBench, a real-session benchmark reconstructed from anonymized and privacy-screened user sessions.<n>The resulting benchmark comprises 200 tasks spanning 8 broad scenarios and 17 fine-grained capability categories.<n>We execute these tasks in isolated Docker containers injected with three forms of real-world environmental complexity: Insufficient, Unstable, and Noisy.
  arXiv  Detail & Related papers  (2026-08-27T02:36:23Z)
• RoboBRIDGE: A Modular Framework for Bridging Policies to Robust Real-World Robotic Agents [26.4490744619718]
  Vision-Language-Action (VLA) models have attracted growing interest as a scalable approach to robotic manipulation.<n>Existing solutions address these limitations individually through model retraining or environment-specific modules.<n>We present RoboBRIDGE, a modular framework that provides an orchestration layer over five coordinated modules.
  arXiv  Detail & Related papers  (2026-07-30T08:55:45Z)
• Exploratory, Communicative, and Deployable: Vision-Driven Embodied Agents for Open-World Mobile Manipulation [47.58516950734307]
  Existing frameworks often rely on privileged simulator states or assume complete instructions, bypassing realistic deployment challenges.<n>We present REAL, an agentic framework for open-world mobile manipulation.<n>We design diverse task compositions to drive data collection, supervised fine-tuning, and online reinforcement learning.
  arXiv  Detail & Related papers  (2026-07-15T09:55:45Z)
• Multi-Agent Robotic Control with Onboard Vision-Language Models [38.78172334790013]
  Vision Language Models (VLMs) and Vision Language Action (VLA) models have shown promise in robotic control.<n>They face significant challenges regarding explainability, generalization, and compute requirements.<n>This paper presents a Multi-Agent System (MAS) architecture that addresses these limitations by deploying specialized agents on onboard hardware.
  arXiv  Detail & Related papers  (2026-07-08T13:37:31Z)
• AgentGym2: Benchmarking Large Language Model Agents in De-Idealized Real-World Environments [84.8170304874972]
  We present AgentGym2, a new evaluation framework with task instances grounded in real-world working demands.<n>It measures agents' ability to execute end-to-end procedures, discover tools via exploration, compose tools for unseen tasks, and remain robust to noisy and underspecified information.
  arXiv  Detail & Related papers  (2026-07-06T14:56:51Z)
• VoLo: A Physical Orchestrator for Open-Vocabulary Long-Horizon Manipulation [32.23131310342716]
  Open-vocabulary long-horizon manipulation requires robots to reason over flexible instructions and complex multi-object scenes.<n>We address these demands with a closed agent loop in which a VLM orchestrates heterogeneous robot capabilities as interruptible tools.<n>We propose VoLoAgent, a VLM that plans, monitors, and recovers by treating a VLA/WAM as an interruptible tool it steers mid-rollout.
  arXiv  Detail & Related papers  (2026-06-05T16:21:27Z)
• AgentHijack: Benchmarking Computer Use Agent Robustness to Common Environment Corruptions [78.49000936275773]
  We introduce AgentHijack, a benchmark designed to evaluate the robustness of computer-use agents under common corruptions.<n>We evaluate a variety of desktop tasks that utilize MLLM-based agents and discover that even minor instances of corruption can result in substantial performance degradation.<n>We propose AgentHijack-Agent, a framework that integrates an action generator with enhanced grounding capabilities and an onlooker responsible for behavior summarization and environment checking.
  arXiv  Detail & Related papers  (2026-05-25T11:09:22Z)
• ManipArena: Comprehensive Real-world Evaluation of Reasoning-Oriented Generalist Robot Manipulation [66.02142169323521]
  Vision-Language-Action models and world models have recently emerged as promising paradigms for general-purpose robotic intelligence.<n>Existing benchmarks are largely simulator-centric, which provide controllability but fail to capture the reality gap caused by perception noise.<n>We introduce ManipArena, a standardized evaluation framework designed to bridge simulation and real-world execution.
  arXiv  Detail & Related papers  (2026-03-30T15:06:41Z)
• MALLVI: A Multi-Agent Framework for Integrated Generalized Robotics Manipulation [0.0]
  MALLVI presents a framework that enables closed-loop feedback driven robotic manipulation.<n>Rather than using a single model, MALLVI coordinates specialized agents to manage perception, localization, reasoning, and high level planning.
  arXiv  Detail & Related papers  (2026-02-18T21:28:56Z)
• UniManip: General-Purpose Zero-Shot Robotic Manipulation with Agentic Operational Graph [23.060488218180936]
  We present UniManip, a framework grounded in a Bi-level Agentic Operational Graph (AOG)<n>By coupling a high-level Agentic Layer for task orchestration with a low-level Scene Layer for dynamic state representation, the system continuously aligns abstract planning with geometric constraints.<n>Experiments validate the system's robust zero-shot capability on unseen objects and tasks, demonstrating a 22.5% and 25.0% higher success rate compared to state-of-the-art VLA and hierarchical baselines.
  arXiv  Detail & Related papers  (2026-02-13T16:47:26Z)
• Agent World Model: Infinity Synthetic Environments for Agentic Reinforcement Learning [62.499592503950026]
  Large language model (LLM) have empowered autonomous agents to perform complex tasks that require multi-turn interactions with tools and environments.<n>We propose Agent World Model (AWM), a fully synthetic environment generation pipeline.<n>We scale to 1,000 environments covering everyday scenarios, in which agents can interact with rich toolsets.
  arXiv  Detail & Related papers  (2026-02-10T18:55:41Z)
• DeepAgent: A General Reasoning Agent with Scalable Toolsets [111.6384541877723]
  DeepAgent is an end-to-end deep reasoning agent that performs autonomous thinking, tool discovery, and action execution.<n>To address the challenges of long-horizon interactions, we introduce an autonomous memory folding mechanism that compresses past interactions into structured episodic, working, and tool memories.<n>We develop an end-to-end reinforcement learning strategy, namely ToolPO, that leverages LLM-simulated APIs and applies tool-call advantage attribution to assign fine-grained credit to the tool invocation tokens.
  arXiv  Detail & Related papers  (2025-10-24T16:24:01Z)
• State and Memory is All You Need for Robust and Reliable AI Agents [29.259008600842517]
  Large language models (LLMs) have enabled powerful advances in natural language understanding and generation.<n>Yet their application to complex, real-world scientific remain limited by challenges in memory, planning, and tool integration.<n>Here, we introduce SciBORG, a modular agentic framework that allows LLM-based agents to autonomously plan, reason, and achieve robust and reliable domain-specific task execution.
  arXiv  Detail & Related papers  (2025-06-30T02:02:35Z)
• From Virtual Agents to Robot Teams: A Multi-Robot Framework Evaluation in High-Stakes Healthcare Context [2.016235597066821]
  Current frameworks treat agents as conceptual task executors rather than physically embodied entities.<n>We propose three design guidelines emphasizing process transparency, proactive failure recovery, and contextual grounding.<n>Our work informs the development of more resilient and robust multi-agent robotic systems.
  arXiv  Detail & Related papers  (2025-06-04T04:05:38Z)
• ThinkGeo: Evaluating Tool-Augmented Agents for Remote Sensing Tasks [64.86209459039313]
  ThinkGeo is an agentic benchmark designed to evaluate tool-augmented agents on remote sensing tasks via structured tool use and multi-step planning.<n>We implement a ReAct-style interaction loop and evaluate both open and closed-source LLMs on 486 structured agentic tasks with 1,773 expert-verified reasoning steps.<n>Our analysis reveals notable disparities in tool accuracy and planning consistency across models.
  arXiv  Detail & Related papers  (2025-05-29T17:59:38Z)
• Interactive Planning Using Large Language Models for Partially Observable Robotics Tasks [54.60571399091711]
  Large Language Models (LLMs) have achieved impressive results in creating robotic agents for performing open vocabulary tasks. We present an interactive planning technique for partially observable tasks using LLMs.
  arXiv  Detail & Related papers  (2023-12-11T22:54:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.