Accelerating the Development of Multimodal, Integrative-AI Systems with Platform for Situated Intelligence

• URL: http://arxiv.org/abs/2010.06084v1
• Date: Mon, 12 Oct 2020 23:53:12 GMT
• Title: Accelerating the Development of Multimodal, Integrative-AI Systems with Platform for Situated Intelligence
• Authors: Sean Andrist and Dan Bohus
• Abstract summary: We describe Platform for Situated Intelligence, an open-source framework for multimodal, integrative-AI systems. We provide a brief, high-level overview of the framework and its main affordances, and discuss its implications for HRI.
• Score: 1.595445991573573
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We describe Platform for Situated Intelligence, an open-source framework for multimodal, integrative-AI systems. The framework provides infrastructure, tools, and components that enable and accelerate the development of applications that process multimodal streams of data and in which timing is critical. The framework is particularly well-suited for developing physically situated interactive systems that perceive and reason about their surroundings in order to better interact with people, such as social robots, virtual assistants, smart meeting rooms, etc. In this paper, we provide a brief, high-level overview of the framework and its main affordances, and discuss its implications for HRI.

Related papers

• Estuary: A Framework For Building Multimodal Low-Latency Real-Time Socially Interactive Agents [0.4711628883579317]
  Estuary is a framework for the development of low-latency, real-time Socially Interactive Agents. It can be run entirely off-cloud to maximize configurability, controllability, of studies, and speed of agent response times.
  arXiv  Detail & Related papers  (2024-10-26T08:08:12Z)
• Internet of Agents: Weaving a Web of Heterogeneous Agents for Collaborative Intelligence [79.5316642687565]
  Existing multi-agent frameworks often struggle with integrating diverse capable third-party agents. We propose the Internet of Agents (IoA), a novel framework that addresses these limitations. IoA introduces an agent integration protocol, an instant-messaging-like architecture design, and dynamic mechanisms for agent teaming and conversation flow control.
  arXiv  Detail & Related papers  (2024-07-09T17:33:24Z)
• SIGMA: An Open-Source Interactive System for Mixed-Reality Task Assistance Research [5.27467559535251]
  We introduce an open-source system called SIGMA as a platform for conducting research on task-assistive agents in mixed-reality scenarios. We present the system's core capabilities, discuss its overall design and implementation, and outline directions for future research enabled by the system.
  arXiv  Detail & Related papers  (2024-05-16T21:21:09Z)
• Position Paper: Agent AI Towards a Holistic Intelligence [53.35971598180146]
  We emphasize developing Agent AI -- an embodied system that integrates large foundation models into agent actions. In this paper, we propose a novel large action model to achieve embodied intelligent behavior, the Agent Foundation Model.
  arXiv  Detail & Related papers  (2024-02-28T16:09:56Z)
• AgentScope: A Flexible yet Robust Multi-Agent Platform [66.64116117163755]
  AgentScope is a developer-centric multi-agent platform with message exchange as its core communication mechanism. The abundant syntactic tools, built-in agents and service functions, user-friendly interfaces for application demonstration and utility monitor, zero-code programming workstation, and automatic prompt tuning mechanism significantly lower the barriers to both development and deployment.
  arXiv  Detail & Related papers  (2024-02-21T04:11:28Z)
• An Interactive Agent Foundation Model [49.77861810045509]
  We propose an Interactive Agent Foundation Model that uses a novel multi-task agent training paradigm for training AI agents. Our training paradigm unifies diverse pre-training strategies, including visual masked auto-encoders, language modeling, and next-action prediction. We demonstrate the performance of our framework across three separate domains -- Robotics, Gaming AI, and Healthcare.
  arXiv  Detail & Related papers  (2024-02-08T18:58:02Z)
• Agent AI: Surveying the Horizons of Multimodal Interaction [83.18367129924997]
  "Agent AI" is a class of interactive systems that can perceive visual stimuli, language inputs, and other environmentally-grounded data. We envision a future where people can easily create any virtual reality or simulated scene and interact with agents embodied within the virtual environment.
  arXiv  Detail & Related papers  (2024-01-07T19:11:18Z)
• Online Learning and Planning in Cognitive Hierarchies [10.28577981317938]
  We extend an existing formal framework to model complex integrated reasoning behaviours of robotic systems. New framework allows for a more flexible modelling of the interactions between different reasoning components.
  arXiv  Detail & Related papers  (2023-10-18T23:53:51Z)
• CSM-H-R: A Context Modeling Framework in Supporting Reasoning Automation for Interoperable Intelligent Systems and Privacy Protection [0.07499722271664144]
  We propose a novel framework for automation of High Level Context (HLC) reasoning across intelligent systems at scale. The design of the framework supports the sharing and inter context among intelligent systems and the components for handling CSMs and the management of hierarchy, relationship, and transition. The implementation of the framework experiments on the HLC reasoning into vector and matrix computing and presents the potential to reach next level of automation.
  arXiv  Detail & Related papers  (2023-08-21T22:21:15Z)
• Platform for Situated Intelligence [19.208956579428296]
  We introduce Platform for Situated Intelligence, an open-source framework created to support the rapid development and study of multimodal, integrative-AI systems. The framework provides infrastructure for sensing, fusing, and making inferences from temporal streams of data across different modalities.
  arXiv  Detail & Related papers  (2021-03-29T22:30:15Z)
• Plato Dialogue System: A Flexible Conversational AI Research Platform [64.82999992143448]
  Plato is a flexible Conversational AI platform written in Python that supports any kind of conversational agent architecture. Plato has been designed to be easy to understand and debug and is agnostic to the underlying learning frameworks that train each component.
  arXiv  Detail & Related papers  (2020-01-17T18:27:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.