Cybo-Waiter: A Physical Agentic Framework for Humanoid Whole-Body Locomotion-Manipulation

• URL: http://arxiv.org/abs/2603.10675v1
• Date: Wed, 11 Mar 2026 11:41:32 GMT
• Title: Cybo-Waiter: A Physical Agentic Framework for Humanoid Whole-Body Locomotion-Manipulation
• Abstract summary: Long horizon execution is challenging for humanoids because locomotion and manipulation are tightly coupled through stance, reachability, and balance.<n>We present a humanoid agent framework that turns VLM plans into verifiable task programs and closes the loop with multi object 3D supervision.<n>Experiments on tabletop manipulation and long horizon humanoid loco manipulation tasks show improved robustness from multi object grounding, temporal stability, and recovery driven replanning.
• Score: 14.648223907591849
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Robots are increasingly expected to execute open ended natural language requests in human environments, which demands reliable long horizon execution under partial observability. This is especially challenging for humanoids because locomotion and manipulation are tightly coupled through stance, reachability, and balance. We present a humanoid agent framework that turns VLM plans into verifiable task programs and closes the loop with multi object 3D geometric supervision. A VLM planner compiles each instruction into a typed JSON sequence of subtasks with explicit predicate based preconditions and success conditions. Using SAM3 and RGB-D, we ground all task relevant entities in 3D, estimate object centroids and extents, and evaluate predicates over stable frames to obtain condition level diagnostics. The supervisor uses these diagnostics to verify subtask completion and to provide condition-level feedback for progression and replanning. We execute each subtask by coordinating humanoid locomotion and whole-body manipulation, selecting feasible motion primitives under reachability and balance constraints. Experiments on tabletop manipulation and long horizon humanoid loco manipulation tasks show improved robustness from multi object grounding, temporal stability, and recovery driven replanning.

Related papers

• Compiling and Benchmarking Task-State Horizons for Embodied Agents [2.8397380902242966]
  Frontier agentic models are increasingly deployed as high-level planners for long-horizon embodied tasks.<n>We define the span of task-relevant state transitions that an agent must track as task-state horizon (TSH)<n>We introduce RoboGraph, a robotic task compiler that translates state-transition dependencies into executable symbolic graphs.
  arXiv  Detail & Related papers  (2026-08-08T09:45:01Z)
• Closing the Loop in Humanoid VLA: Persistent 3D Object Tokens for Verifiable Loco-Manipulation [14.431337587744538]
  Long-horizon humanoid loco-manipulation requires the robot to treat task objects as persistent physical entities across movement, contact, and recovery.<n>We propose emphPersistent Object Tokenization (POT), which maintains role-indexed 3D object records from RGB-D observations.<n>Instantiated as emphPOT-VLA, the same object records condition action generation and support geometric predicate checks, yielding a closed-loop execution system.
  arXiv  Detail & Related papers  (2026-07-20T14:52:46Z)
• EnactToM: An Evolving Benchmark for Functional Theory of Mind in Embodied Agents [75.01735520608075]
  Existing benchmarks mostly test literal Theory of Mind (ToM) by asking direct belief questions.<n>We introduce EnactToM, an evolving benchmark of 300 embodied multi-agent tasks set in a 3D household.<n>On the hard split, all seven evaluated frontier models score 0.0% Pass3 on functional task completion, while averaging 45.0% on literal belief probes.
  arXiv  Detail & Related papers  (2026-05-11T00:04:19Z)
• Melding LLM and temporal logic for reliable human-swarm collaboration in complex scenarios [17.56189537207367]
  We introduce a neuro-symbolic framework for long-horizon human-swarm collaboration.<n>We formalize mission goals and operational rules as temporal logic formulas and admissible task orderings as task automata.<n>An event-triggered interaction protocol limits operator involvement to sparse, high-level confirmation and guidance.
  arXiv  Detail & Related papers  (2026-05-08T15:32:14Z)
• SceneTeract: Agentic Functional Affordances and VLM Grounding in 3D Scenes [49.117927512531715]
  We introduce SceneTeract, a framework that verifies 3D scene functionality under agent-specific constraints.<n> SceneTeract decomposes complex activities into sequences of atomic actions and validates each step against accessibility requirements.<n>We release the SceneTeract verification suite and data to bridge perception and physical reality in embodied 3D scene understanding.
  arXiv  Detail & Related papers  (2026-03-31T14:31:18Z)
• RADAR: Closed-Loop Robotic Data Generation via Semantic Planning and Autonomous Causal Environment Reset [48.645870795753105]
  We introduce Robust Autonomous Data Acquisition for Robotics (RADAR)<n>RADAR is a fully autonomous, closed-loop data generation engine that completely removes human intervention from the collection cycle.<n>In simulation, our framework achieves up to 90% success rates on complex, long-horizon tasks.
  arXiv  Detail & Related papers  (2026-03-12T11:18:52Z)
• HAIC: Humanoid Agile Object Interaction Control via Dynamics-Aware World Model [56.4392302336014]
  We present HAIC, a framework for robust interaction across diverse object dynamics without external state estimation.<n>Our key contribution is a dynamics predictor that estimates high-order object states (velocity, acceleration) solely from proprioceptive history.<n> Experiments on a humanoid robot show HAIC achieves high success rates in agile tasks.
  arXiv  Detail & Related papers  (2026-02-12T09:34:35Z)
• Automatic Cognitive Task Generation for In-Situ Evaluation of Embodied Agents [43.01384379901339]
  We propose a dynamic in-situ task generation method for unseen environments inspired by human cognition.<n>In the interaction stage, the agent actively interacts with the environment, creating a loop between task execution and generation.<n>Experiments across 10 unseen scenes demonstrate that TEA automatically generated 87,876 tasks in two cycles.
  arXiv  Detail & Related papers  (2026-02-05T03:07:00Z)
• PILOT: A Perceptive Integrated Low-level Controller for Loco-manipulation over Unstructured Scenes [18.916978556216097]
  PILOT is a single-stage reinforcement learning framework tailored for perceptive loco-manipulation.<n>It synergizes perceptive locomotion and expansive whole-body control within a single policy.<n> PILOT demonstrates superior stability, command tracking precision, and terrain traversability compared to existing baselines.
  arXiv  Detail & Related papers  (2026-01-24T12:25:20Z)
• Generalizable Geometric Prior and Recurrent Spiking Feature Learning for Humanoid Robot Manipulation [90.90219129619344]
  This paper presents a novel R-prior-S, Recurrent Geometric-priormodal Policy with Spiking features.<n>To ground high-level reasoning in physical reality, we leverage lightweight 2D geometric inductive biases.<n>For the data efficiency issue in robotic action generation, we introduce a Recursive Adaptive Spiking Network.
  arXiv  Detail & Related papers  (2026-01-13T23:36:30Z)
• Executable Analytic Concepts as the Missing Link Between VLM Insight and Precise Manipulation [70.8381970762877]
  Vision-Language Models (VLMs) have demonstrated remarkable capabilities in semantic reasoning and task planning.<n>We introduce GRACE, a novel framework that grounds VLM-based reasoning through executable analytic concepts.<n>G GRACE provides a unified and interpretable interface between high-level instruction understanding and low-level robot control.
  arXiv  Detail & Related papers  (2025-10-09T09:08:33Z)
• HumanoidVerse: A Versatile Humanoid for Vision-Language Guided Multi-Object Rearrangement [51.16740261131198]
  We introduce HumanoidVerse, a novel framework for vision-language guided humanoid control.<n>HumanoidVerse supports consecutive manipulation of multiple objects, guided only by natural language instructions and egocentric camera RGB observations.<n>Our work represents a key step toward robust, general-purpose humanoid agents capable of executing complex, sequential tasks under real-world sensory constraints.
  arXiv  Detail & Related papers  (2025-08-23T08:23:14Z)
• ODYSSEY: Open-World Quadrupeds Exploration and Manipulation for Long-Horizon Tasks [46.676862567167625]
  ODYSSEY is a unified mobile manipulation framework for agile quadruped robots equipped with manipulators.<n>To address the challenge of egocentric perception in language-conditioned tasks, we introduce a hierarchical planner powered by a vision-language model.<n>At the control level, our novel whole-body policy achieves robust coordination across challenging terrains.
  arXiv  Detail & Related papers  (2025-08-11T17:54:31Z)
• Embodied Long Horizon Manipulation with Closed-loop Code Generation and Incremental Few-shot Adaptation [12.077740860502878]
  Embodied long-horizon manipulation requires robotic systems to process multimodal inputs-such as vision and natural language-and translate them into executable actions.<n>Recent methods have explored using large language models (LLMs) as high-level planners that decompose tasks into subtasks using natural language and guide pretrained low-level controllers.<n>Our framework achieves state-of-the-art performance on 30+ diverse seen and unseen long-horizon tasks across LoHoRavens, CALVIN, Franka Kitchen, and cluttered real-world settings.
  arXiv  Detail & Related papers  (2025-03-27T20:32:58Z)
• SAM-E: Leveraging Visual Foundation Model with Sequence Imitation for Embodied Manipulation [62.58480650443393]
  Segment Anything (SAM) is a vision-foundation model for generalizable scene understanding and sequence imitation. We develop a novel multi-channel heatmap that enables the prediction of the action sequence in a single pass.
  arXiv  Detail & Related papers  (2024-05-30T00:32:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.