AgentBay: A Hybrid Interaction Sandbox for Seamless Human-AI Intervention in Agentic Systems

• URL: http://arxiv.org/abs/2512.04367v1
• Date: Thu, 04 Dec 2025 01:31:00 GMT
• Title: AgentBay: A Hybrid Interaction Sandbox for Seamless Human-AI Intervention in Agentic Systems
• Authors: Yun Piao, Hongbo Min, Hang Su, Leilei Zhang, Lei Wang, Yue Yin, Xiao Wu, Zhejing Xu, Liwei Qu, Hang Li, Xinxin Zeng, Wei Tian, Fei Yu, Xiaowei Li, Jiayi Jiang, Tongxu Liu, Hao Tian, Yufei Que, Xiaobing Tu, Bing Suo, Yuebing Li, Xiangting Chen, Zeen Zhao, Jiaming Tang, Wei Huang, Xuguang Li, Jing Zhao, Jin Li, Jie Shen, Jinkui Ren, Xiantao Zhang,
• Abstract summary: We present AgentBay, a novel service designed from ground up for hybrid interaction.<n>AgentBay provides secure, isolated execution environments Windows, Linux, Android, Web Browsers, and interpreters.<n>Our evaluation demonstrates strong results in security, performance, and task completion rates.
• Score: 29.451397580654316
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The rapid advancement of Large Language Models (LLMs) is catalyzing a shift towards autonomous AI Agents capable of executing complex, multi-step tasks. However, these agents remain brittle when faced with real-world exceptions, making Human-in-the-Loop (HITL) supervision essential for mission-critical applications. In this paper, we present AgentBay, a novel sandbox service designed from the ground up for hybrid interaction. AgentBay provides secure, isolated execution environments spanning Windows, Linux, Android, Web Browsers, and Code interpreters. Its core contribution is a unified session accessible via a hybrid control interface: An AI agent can interact programmatically via mainstream interfaces (MCP, Open Source SDK), while a human operator can, at any moment, seamlessly take over full manual control. This seamless intervention is enabled by Adaptive Streaming Protocol (ASP). Unlike traditional VNC/RDP, ASP is specifically engineered for this hybrid use case, delivering an ultra-low-latency, smoother user experience that remains resilient even in weak network environments. It achieves this by dynamically blending command-based and video-based streaming, adapting its encoding strategy based on network conditions and the current controller (AI or human). Our evaluation demonstrates strong results in security, performance, and task completion rates. In a benchmark of complex tasks, the AgentBay (Agent + Human) model achieved more than 48% success rate improvement. Furthermore, our ASP protocol reduces bandwidth consumption by up to 50% compared to standard RDP, and in end-to-end latency with around 5% reduction, especially under poor network conditions. We posit that AgentBay provides a foundational primitive for building the next generation of reliable, human-supervised autonomous systems.

Related papers

• Toward Autonomous O-RAN: A Multi-Scale Agentic AI Framework for Real-Time Network Control and Management [39.17062930275755]
  This article proposes a multi-scale agentic AI framework for Open Radio Access Networks (O-RAN)<n>It organizes RAN intelligence as a coordinated hierarchy across the Non-Real-Time (Non-RT), Near-Real-Time (Near-RT), and Real-Time (RT) control loops.<n>We show how these agents cooperate through standardized O-RAN interfaces and telemetry.
  arXiv  Detail & Related papers  (2026-02-15T12:34:01Z)
• Agent World Model: Infinity Synthetic Environments for Agentic Reinforcement Learning [62.499592503950026]
  Large language model (LLM) have empowered autonomous agents to perform complex tasks that require multi-turn interactions with tools and environments.<n>We propose Agent World Model (AWM), a fully synthetic environment generation pipeline.<n>We scale to 1,000 environments covering everyday scenarios, in which agents can interact with rich toolsets.
  arXiv  Detail & Related papers  (2026-02-10T18:55:41Z)
• ComAgent: Multi-LLM based Agentic AI Empowered Intelligent Wireless Networks [62.031889234230725]
  6G networks rely on complex cross-layer optimization.<n> manually translating high-level intents into mathematical formulations remains a bottleneck.<n>We present ComAgent, a multi-LLM agentic AI framework.
  arXiv  Detail & Related papers  (2026-01-27T13:43:59Z)
• Trusted AI Agents in the Cloud [4.2366483628004366]
  Omega is a system that enables trusted AI agents by enforcing end-to-end isolation.<n>It provides efficient multi-agent orchestration with cross-principal trust establishment.<n>It achieves high performance while enabling high-density, policy-compliant multi-agent deployments at cloud scale.
  arXiv  Detail & Related papers  (2025-12-05T18:48:53Z)
• UltraCUA: A Foundation Model for Computer Use Agents with Hybrid Action [77.63125913907771]
  We present UltraCUA, a foundation model that bridges the gap between GUI primitives and high-level programmatic tool calls.<n>Experiments with our 7B and 32B models demonstrate substantial improvements over state-of-the-art agents.
  arXiv  Detail & Related papers  (2025-10-20T17:48:26Z)
• VeriOS: Query-Driven Proactive Human-Agent-GUI Interaction for Trustworthy OS Agents [39.3943822850841]
  We introduce VeriOS-Agent, a trustworthy OS agent trained with a two-stage learning paradigm.<n>We show that VeriOS-Agent improves the average step-wise success rate by 20.64% in untrustworthy scenarios over the state-of-the-art.
  arXiv  Detail & Related papers  (2025-09-09T09:46:01Z)
• Toward Super Agent System with Hybrid AI Routers [19.22599167969104]
  Super agents can fulfill diverse user needs, such as summarization, coding, and research.<n>This paper presents a design of the Super Agent System powered by the hybrid AI routers.<n>With advances in multi-modality models and edge hardware, we envision that most computations can be handled locally, with cloud collaboration only as needed.
  arXiv  Detail & Related papers  (2025-04-11T00:54:56Z)
• Internet of Agents: Weaving a Web of Heterogeneous Agents for Collaborative Intelligence [79.5316642687565]
  Existing multi-agent frameworks often struggle with integrating diverse capable third-party agents. We propose the Internet of Agents (IoA), a novel framework that addresses these limitations. IoA introduces an agent integration protocol, an instant-messaging-like architecture design, and dynamic mechanisms for agent teaming and conversation flow control.
  arXiv  Detail & Related papers  (2024-07-09T17:33:24Z)
• SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering [79.07755560048388]
  SWE-agent is a system that facilitates LM agents to autonomously use computers to solve software engineering tasks. SWE-agent's custom agent-computer interface (ACI) significantly enhances an agent's ability to create and edit code files, navigate entire repositories, and execute tests and other programs. We evaluate SWE-agent on SWE-bench and HumanEvalFix, achieving state-of-the-art performance on both with a pass@1 rate of 12.5% and 87.7%, respectively.
  arXiv  Detail & Related papers  (2024-05-06T17:41:33Z)
• AgentScope: A Flexible yet Robust Multi-Agent Platform [66.64116117163755]
  AgentScope is a developer-centric multi-agent platform with message exchange as its core communication mechanism. The abundant syntactic tools, built-in agents and service functions, user-friendly interfaces for application demonstration and utility monitor, zero-code programming workstation, and automatic prompt tuning mechanism significantly lower the barriers to both development and deployment.
  arXiv  Detail & Related papers  (2024-02-21T04:11:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.