Asynchronous Tool Usage for Real-Time Agents

• URL: http://arxiv.org/abs/2410.21620v1
• Date: Mon, 28 Oct 2024 23:57:19 GMT
• Title: Asynchronous Tool Usage for Real-Time Agents
• Authors: Antonio A. Ginart, Naveen Kodali, Jason Lee, Caiming Xiong, Silvio Savarese, John Emmons,
• Abstract summary: We introduce asynchronous AI agents capable of parallel processing and real-time tool-use. Our key contribution is an event-driven finite-state machine architecture for agent execution and prompting. This work presents both a conceptual framework and practical tools for creating AI agents capable of fluid, multitasking interactions.
• Score: 61.3041983544042
• License:
• Abstract: While frontier large language models (LLMs) are capable tool-using agents, current AI systems still operate in a strict turn-based fashion, oblivious to passage of time. This synchronous design forces user queries and tool-use to occur sequentially, preventing the systems from multitasking and reducing interactivity. To address this limitation, we introduce asynchronous AI agents capable of parallel processing and real-time tool-use. Our key contribution is an event-driven finite-state machine architecture for agent execution and prompting, integrated with automatic speech recognition and text-to-speech. Drawing inspiration from the concepts originally developed for real-time operating systems, this work presents both a conceptual framework and practical tools for creating AI agents capable of fluid, multitasking interactions.

Related papers

• Internet of Agents: Weaving a Web of Heterogeneous Agents for Collaborative Intelligence [79.5316642687565]
  Existing multi-agent frameworks often struggle with integrating diverse capable third-party agents. We propose the Internet of Agents (IoA), a novel framework that addresses these limitations. IoA introduces an agent integration protocol, an instant-messaging-like architecture design, and dynamic mechanisms for agent teaming and conversation flow control.
  arXiv  Detail & Related papers  (2024-07-09T17:33:24Z)
• ROS-LLM: A ROS framework for embodied AI with task feedback and structured reasoning [74.58666091522198]
  We present a framework for intuitive robot programming by non-experts. We leverage natural language prompts and contextual information from the Robot Operating System (ROS) Our system integrates large language models (LLMs), enabling non-experts to articulate task requirements to the system through a chat interface.
  arXiv  Detail & Related papers  (2024-06-28T08:28:38Z)
• AgentScope: A Flexible yet Robust Multi-Agent Platform [66.64116117163755]
  AgentScope is a developer-centric multi-agent platform with message exchange as its core communication mechanism. The abundant syntactic tools, built-in agents and service functions, user-friendly interfaces for application demonstration and utility monitor, zero-code programming workstation, and automatic prompt tuning mechanism significantly lower the barriers to both development and deployment.
  arXiv  Detail & Related papers  (2024-02-21T04:11:28Z)
• LLMind: Orchestrating AI and IoT with LLM for Complex Task Execution [18.816077341295628]
  We present LLMind, a task-oriented AI framework that enables effective collaboration among IoT devices. Inspired by the functional specialization theory of the brain, our framework integrates an LLM with domain-specific AI modules. Complex tasks, which may involve collaborations of multiple domain-specific AI modules and IoT devices, are executed through a control script.
  arXiv  Detail & Related papers  (2023-12-14T14:57:58Z)
• RED: A Systematic Real-Time Scheduling Approach for Robotic Environmental Dynamics [11.38746414146899]
  We introduce RED, a systematic real-time scheduling approach designed to support multi-task deep neural network workloads in resource-limited robotic systems. It is designed to adaptively manage the Robotic Environmental Dynamics (RED) while adhering to real-time constraints.
  arXiv  Detail & Related papers  (2023-08-29T15:04:08Z)
• ART: Automatic multi-step reasoning and tool-use for large language models [105.57550426609396]
  Large language models (LLMs) can perform complex reasoning in few- and zero-shot settings. Each reasoning step can rely on external tools to support computation beyond the core LLM capabilities. We introduce Automatic Reasoning and Tool-use (ART), a framework that uses frozen LLMs to automatically generate intermediate reasoning steps as a program.
  arXiv  Detail & Related papers  (2023-03-16T01:04:45Z)
• Realistic simulation of users for IT systems in cyber ranges [63.20765930558542]
  We instrument each machine by means of an external agent to generate user activity. This agent combines both deterministic and deep learning based methods to adapt to different environment. We also propose conditional text generation models to facilitate the creation of conversations and documents.
  arXiv  Detail & Related papers  (2021-11-23T10:53:29Z)
• Workflow Automation for Cyber Physical System Development Processes [1.6735240552964108]
  Development of Cyber Physical Systems (CPSs) requires close interaction between developers with expertise in many domains. We introduce a workflow modeling language for the automation of complex CPS development processes. We implement a platform for execution of these models in the Assurance-based Learning-enabled CPS Toolchain.
  arXiv  Detail & Related papers  (2020-04-12T17:32:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.