Advancing Omnimodal Embodied Agents from Isolated Skills to Everyday Physical Autonomy

• URL: http://arxiv.org/abs/2606.27251v1
• Date: Thu, 25 Jun 2026 16:36:35 GMT
• Title: Advancing Omnimodal Embodied Agents from Isolated Skills to Everyday Physical Autonomy
• Abstract summary: Building persistent embodied agents in unstructured environments requires unified orchestration of heterogeneous tools.<n>We present OmniAct, a framework integrating a multimodal semantic planner for skill routing across unified action spaces.<n>It achieves consistent improvements in end-to-end success across all complexity levels.
• Score: 79.09714980186278
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Building persistent embodied agents in unstructured environments demands unified orchestration of heterogeneous tools spanning both cyber (APIs, IoT) and physical (manipulation, navigation) domains, coupled with autonomous recovery from physical failures that inevitably arise over extended operation. Existing systems treat these as separate problems: VLM-based planners lack a unified cyber-physical action space, agent frameworks accumulate unbounded context that degrades temporal coherence, and VLA policies execute open-loop without detecting their own failures. We argue that persistent autonomy requires not a monolithic model but a hierarchical asynchronous architecture with explicit separation of planning, memory, and verification. To this end, we present OmniAct, a framework integrating a multimodal semantic planner for skill routing across unified action spaces, an adaptive hierarchical memory with event-boundary-driven compression for sub-linear context growth, and an asynchronous visual preemption engine that closes the semantic loop during physical execution. Across 40 real-world long-horizon tasks on two robotic platforms coordinating four IoT devices, OmniAct achieves consistent improvements in end-to-end success across all complexity levels, maintains near-flat token consumption over under 100k+ accumulated interaction tokens, and elevates mid-scale open-weight models to proprietary-level performance.

Related papers

• Learning Reactive Dexterous Grasping via Hierarchical Task-Space RL Planning and Joint-Space QP Control [50.28263951510334]
  We propose a hybrid hierarchical control framework for reactive dexterous grasping.<n>The proposed framework explicitly decouples high-level spatial intent from low-level joint execution.<n>We extensively validate the proposed framework through a rigorous simulation-to-reality pipeline.
  arXiv  Detail & Related papers  (2026-05-05T04:49:38Z)
• DexWorldModel: Causal Latent World Modeling towards Automated Learning of Embodied Tasks [54.32016216994156]
  We introduce the Causal Latent World Model (CLWM), which employs DINOv3 features as generative targets to disentangle interaction semantics from visual noise.<n>To overcome memory scaling, CLWM features a Dual-State Test-Time Training (TTT) Memory that guarantees a strict $mathcalO(1)$ footprint for long-horizon tasks.<n>We present EmbodiChain, an online framework that establishes the Efficiency Law by injecting an infinite flow of physics-grounded trajectories during training.
  arXiv  Detail & Related papers  (2026-04-13T03:19:36Z)
• V-CAGE: Vision-Closed-Loop Agentic Generation Engine for Robotic Manipulation [6.820118518027692]
  We present V-CAGE, an agentic framework for autonomous robotic data synthesis.<n>Unlike traditional scripted pipelines, V-CAGE operates as an embodied agentic system.<n>To overcome the storage bottleneck of massive video datasets, we implement a perceptually-driven compression algorithm.
  arXiv  Detail & Related papers  (2026-04-10T06:56:17Z)
• ROSClaw: A Hierarchical Semantic-Physical Framework for Heterogeneous Multi-Agent Collaboration [9.434833852972353]
  ROSClaw is an agent framework for heterogeneous robots that integrates policy learning and task execution within a unified vision-language model (VLM) controller.<n>By establishing an autonomous closedloop framework, ROSClaw minimizes the reliance on robot-specific development.
  arXiv  Detail & Related papers  (2026-04-06T13:16:24Z)
• RADAR: Closed-Loop Robotic Data Generation via Semantic Planning and Autonomous Causal Environment Reset [48.645870795753105]
  We introduce Robust Autonomous Data Acquisition for Robotics (RADAR)<n>RADAR is a fully autonomous, closed-loop data generation engine that completely removes human intervention from the collection cycle.<n>In simulation, our framework achieves up to 90% success rates on complex, long-horizon tasks.
  arXiv  Detail & Related papers  (2026-03-12T11:18:52Z)
• ACLM: ADMM-Based Distributed Model Predictive Control for Collaborative Loco-Manipulation [9.708461585583791]
  Collaborative transportation of heavy payloads via loco-manipulation is a challenging yet essential capability for legged robots.<n>This study proposes an Alternating Direction Method of Multipliers-based distributed model predictive control framework for loco-manipulation.
  arXiv  Detail & Related papers  (2026-03-07T08:06:51Z)
• UniManip: General-Purpose Zero-Shot Robotic Manipulation with Agentic Operational Graph [23.060488218180936]
  We present UniManip, a framework grounded in a Bi-level Agentic Operational Graph (AOG)<n>By coupling a high-level Agentic Layer for task orchestration with a low-level Scene Layer for dynamic state representation, the system continuously aligns abstract planning with geometric constraints.<n>Experiments validate the system's robust zero-shot capability on unseen objects and tasks, demonstrating a 22.5% and 25.0% higher success rate compared to state-of-the-art VLA and hierarchical baselines.
  arXiv  Detail & Related papers  (2026-02-13T16:47:26Z)
• Towards Efficient Agents: A Co-Design of Inference Architecture and System [66.59916327634639]
  This paper presents AgentInfer, a unified framework for end-to-end agent acceleration.<n>We decompose the problem into four synergistic components: AgentCollab, AgentSched, AgentSAM, and AgentCompress.<n>Experiments on the BrowseComp-zh and DeepDiver benchmarks demonstrate that through the synergistic collaboration of these methods, AgentInfer reduces ineffective token consumption by over 50%.
  arXiv  Detail & Related papers  (2025-12-20T12:06:13Z)
• AutoLayout: Closed-Loop Layout Synthesis via Slow-Fast Collaborative Reasoning [102.71841660031065]
  Auto is a fully automated method that integrates a closed-loop self-validation process within a dual-system framework.<n>The effectiveness of Auto was validated across 8 distinct scenarios, where it demonstrated a significant 10.1% improvement over SOTA methods.
  arXiv  Detail & Related papers  (2025-07-06T08:35:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.