Octopus Protocol: One-Shot Hardware Discovery and Control for AI Agents via Infrastructure-as-Prompts

• URL: http://arxiv.org/abs/2605.09055v1
• Date: Sat, 09 May 2026 16:57:11 GMT
• Title: Octopus Protocol: One-Shot Hardware Discovery and Control for AI Agents via Infrastructure-as-Prompts
• Abstract summary: We present Octopus Protocol, a system that collapses that cost to a single shell command.<n>Two architectural principles make this work: protocols are prompts, not code, and the coding agent is the runtime.<n>One command onboards the hardware in 10-15 minutes and exposes up to 30 MCP tools; an MCP-compliant client then performs closed-loop visual-motor control through tools no human wrote.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent agentic-robotics systems, from Code-asPolicies to modern vision-language-action (VLA) foundation models, presuppose that drivers, SDKs, or ROS-style primitives for the target hardware already exist. Writing those primitives is the dominant engineering cost of bringing up new hardware for agent control. We present Octopus Protocol, a system that collapses that cost to a single shell command. Given only raw OS access and a language-model API key, a coding agent executes a five-stage pipeline--PROBE, IDENTIFY, INTERFACE, SERVE, DEPLOY--to discover connected devices, infer their capabilities, generate a Model Context Protocol (MCP) server with typed tools, and deploy it as a live HTTP endpoint. A persistent daemon then monitors the system, heals broken code, and perceives physical state through the camera tools it generated for itself. Two architectural principles make this work: protocols are prompts, not code, and the coding agent is the runtime. We validate the system on three heterogeneous platforms (PC/WSL, Apple Silicon macOS, Raspberry Pi 4) and on a commercial 6-DOF robotic arm with USB camera feedback. One command onboards the hardware in ~10-15 minutes and exposes up to 30 MCP tools; an MCP-compliant client then performs closed-loop visual-motor control through tools no human wrote.

Related papers

• A hardware-safety-gated system for LLM-written native ARTIQ control code on a trapped-ion platform [18.226528651110133]
  Large-language-model (LLM) agents can write and run experimental control code.<n>Unchecked code can damage the apparatus, and there is no formal, per-operation boundary between human authorization/supervision, and agent decisions.<n>We present a control system that places an LLM agent in the loop of a trapped-ion experiment while enforcing such a boundary.
  arXiv  Detail & Related papers  (2026-06-25T16:19:23Z)
• Agents All the Way Down; A Methodology for Building Custom AI Agents from Substrate to Production [1.0499611180329804]
  No published practice sets out how to build one end to end.<n>This paper writes that practice down as a methodology, Agents All the Way Down.<n>It was distilled from the AAC, a custom agent for the open-source LAMB platform.
  arXiv  Detail & Related papers  (2026-06-10T09:44:54Z)
• Workstream: A Local-First Developer Command Center for the AI-Augmented Engineering Workflow [0.0]
  We present Workstream, an open-source, local-first developer command center that aggregates pull requests, task management, calendar, AI-powered code review, historical review intelligence, repository AI-readiness scoring, and agent observability into a single interface.<n>We describe the system architecture, a novel 5-category AI readiness scoring algorithm, a review intelligence pipeline that mines historical PR reviews for team-specific patterns, and an agent observability layer implementing the Model Context Protocol (MCP), Agent-to-Agent (A2A), and Agent Observability Protocol (AOP)
  arXiv  Detail & Related papers  (2026-04-18T16:24:29Z)
• Dive into Claude Code: The Design Space of Today's and Future AI Agent Systems [42.345465506597044]
  Claude Code is an agentic coding tool that can run shell commands, edit files, and call external services on behalf of the user.<n>This study describes its comprehensive architecture by analyzing the publicly available TypeScript source code.
  arXiv  Detail & Related papers  (2026-04-14T17:59:37Z)
• VLAgents: A Policy Server for Efficient VLA Inference [10.731074886477515]
  We introduce VLAgents, a modular policy server that abstracts VLA inferencing behind a unified Gymnasium-style protocol.<n>Its communication layer transparently adapts to the context by supporting both zero-copy shared memory for high-speed simulation and compressed streaming for remote hardware.<n>In a benchmark with both local and remote communication, we demonstrate how it outperforms the default policy servers provided by OpenVLA, OpenPi, and LeRobot.
  arXiv  Detail & Related papers  (2026-01-16T12:58:59Z)
• Code2MCP: Transforming Code Repositories into MCP Services [53.234097255779744]
  Model Context Protocol (MCP) aims to create a standard for how Large Language Models use tools.<n>We introduce Code2MCP, an agent-based framework that automatically transforms a GitHub repository into a functional MCP service.<n>We demonstrate that Code2MCP successfully transforms open-source computing libraries in scientific fields such as bioinformatics, mathematics, and fluid dynamics.
  arXiv  Detail & Related papers  (2025-09-07T06:13:25Z)
• OS Agents: A Survey on MLLM-based Agents for General Computing Devices Use [101.57043903478257]
  The dream to create AI assistants as capable and versatile as the fictional J.A.R.V.I.S from Iron Man has long captivated imaginations.<n>With the evolution of (multi-modal) large language models ((M)LLMs), this dream is closer to reality.<n>This survey aims to consolidate the state of OS Agents research, providing insights to guide both academic inquiry and industrial development.
  arXiv  Detail & Related papers  (2025-08-06T14:33:45Z)
• LiteCUA: Computer as MCP Server for Computer-Use Agent on AIOS [41.43108796746488]
  AIOS 1.0 is a novel platform designed to advance computer-use agent capabilities through environmental contextualization.<n>We introduce LiteCUA, a lightweight computer-use agent built on AIOS 1.0 that achieves a 14.66% success rate on the OSWorld benchmark.
  arXiv  Detail & Related papers  (2025-05-24T18:56:00Z)
• UFO2: The Desktop AgentOS [60.317812905300336]
  UFO2 is a multiagent AgentOS for Windows desktops that elevates into practical, system-level automation.<n>We evaluate UFO2 across over 20 real-world Windows applications, demonstrating substantial improvements in robustness and execution accuracy over prior CUAs.<n>Our results show that deep OS integration unlocks a scalable path toward reliable, user-aligned desktop automation.
  arXiv  Detail & Related papers  (2025-04-20T13:04:43Z)
• PC-Agent: A Hierarchical Multi-Agent Collaboration Framework for Complex Task Automation on PC [98.82146219495792]
  In this paper, we propose a hierarchical agent framework named PC-Agent.<n>From the perception perspective, we devise an Active Perception Module (APM) to overcome the inadequate abilities of current MLLMs in perceiving screenshot content.<n>From the decision-making perspective, to handle complex user instructions and interdependent subtasks more effectively, we propose a hierarchical multi-agent collaboration architecture.
  arXiv  Detail & Related papers  (2025-02-20T05:41:55Z)
• AgentKit: Structured LLM Reasoning with Dynamic Graphs [91.09525140733987]
  We propose an intuitive LLM prompting framework (AgentKit) for multifunctional agents. AgentKit offers a unified framework for explicitly constructing a complex "thought process" from simple natural language prompts.
  arXiv  Detail & Related papers  (2024-04-17T15:40:45Z)
• ScreenAgent: A Vision Language Model-driven Computer Control Agent [17.11085071288194]
  We build an environment for a Vision Language Model (VLM) agent to interact with a real computer screen. Within this environment, the agent can observe screenshots and manipulate the Graphics User Interface (GUI) by outputting mouse and keyboard actions. We construct the ScreenAgent dataset, which collects screenshots and action sequences when completing a variety of daily computer tasks.
  arXiv  Detail & Related papers  (2024-02-09T02:33:45Z)
• Agents: An Open-source Framework for Autonomous Language Agents [98.91085725608917]
  We consider language agents as a promising direction towards artificial general intelligence. We release Agents, an open-source library with the goal of opening up these advances to a wider non-specialist audience.
  arXiv  Detail & Related papers  (2023-09-14T17:18:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.