CyberCortex.AI: An AI-based Operating System for Autonomous Robotics and Complex Automation

• URL: http://arxiv.org/abs/2409.01241v3
• Date: Fri, 4 Oct 2024 11:32:08 GMT
• Title: CyberCortex.AI: An AI-based Operating System for Autonomous Robotics and Complex Automation
• Authors: Sorin Grigorescu, Mihai Zaha,
• Abstract summary: We introduce CyberCortex AI, a robotics OS designed to enable heterogeneous AI-based robotics and complex automation applications. CyberCortex AI is a decentralized distributed OS which enables robots to talk to each other, as well as to High Performance Computers in the cloud. Sensory and control data from the robots is streamed towards HPC systems with the purpose of training AI algorithms, which are afterwards deployed on the robots.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The underlying framework for controlling autonomous robots and complex automation applications are Operating Systems (OS) capable of scheduling perception-and-control tasks, as well as providing real-time data communication to other robotic peers and remote cloud computers. In this paper, we introduce CyberCortex AI, a robotics OS designed to enable heterogeneous AI-based robotics and complex automation applications. CyberCortex AI is a decentralized distributed OS which enables robots to talk to each other, as well as to High Performance Computers (HPC) in the cloud. Sensory and control data from the robots is streamed towards HPC systems with the purpose of training AI algorithms, which are afterwards deployed on the robots. Each functionality of a robot (e.g. sensory data acquisition, path planning, motion control, etc.) is executed within a so-called DataBlock of Filters shared through the internet, where each filter is computed either locally on the robot itself, or remotely on a different robotic system. The data is stored and accessed via a so-called Temporal Addressable Memory (TAM), which acts as a gateway between each filter's input and output. CyberCortex AI has two main components: i) the CyberCortex AI inference system, which is a real-time implementation of the DataBlock running on the robots' embedded hardware, and ii) the CyberCortex AI dojo, which runs on an HPC computer in the cloud, and it is used to design, train and deploy AI algorithms. We present a quantitative and qualitative performance analysis of the proposed approach using two collaborative robotics applications: i) a forest fires prevention system based on an Unitree A1 legged robot and an Anafi Parrot 4K drone, as well as ii) an autonomous driving system which uses CyberCortex AI for collaborative perception and motion control.

Related papers

• $π_0$: A Vision-Language-Action Flow Model for General Robot Control [77.32743739202543]
  We propose a novel flow matching architecture built on top of a pre-trained vision-language model (VLM) to inherit Internet-scale semantic knowledge. We evaluate our model in terms of its ability to perform tasks in zero shot after pre-training, follow language instructions from people, and its ability to acquire new skills via fine-tuning.
  arXiv  Detail & Related papers  (2024-10-31T17:22:30Z)
• RoboScript: Code Generation for Free-Form Manipulation Tasks across Real and Simulation [77.41969287400977]
  This paper presents textbfRobotScript, a platform for a deployable robot manipulation pipeline powered by code generation. We also present a benchmark for a code generation benchmark for robot manipulation tasks in free-form natural language. We demonstrate the adaptability of our code generation framework across multiple robot embodiments, including the Franka and UR5 robot arms.
  arXiv  Detail & Related papers  (2024-02-22T15:12:00Z)
• LPAC: Learnable Perception-Action-Communication Loops with Applications to Coverage Control [80.86089324742024]
  We propose a learnable Perception-Action-Communication (LPAC) architecture for the problem. CNN processes localized perception; a graph neural network (GNN) facilitates robot communications. Evaluations show that the LPAC models outperform standard decentralized and centralized coverage control algorithms.
  arXiv  Detail & Related papers  (2024-01-10T00:08:00Z)
• Knowledge-Driven Robot Program Synthesis from Human VR Demonstrations [16.321053835017942]
  We present a system for automatically generating executable robot control programs from human task demonstrations in virtual reality (VR) We leverage common-sense knowledge and game engine-based physics to semantically interpret human VR demonstrations. We demonstrate our approach in the context of force-sensitive fetch-and-place for a robotic shopping assistant.
  arXiv  Detail & Related papers  (2023-06-05T09:37:53Z)
• Decentralised construction of a global coordinate system in a large swarm of minimalistic robots [0.8701566919381223]
  In this study, we present an algorithm to enable positional self-awareness in a swarm of minimalistic error-prone robots. Despite being unable to measure the bearing of incoming messages, the robots running our algorithm can calculate their position within a swarm deployed in a regular formation. Our solution has fewer requirements than state-of-the-art algorithms and contains collective noise-filtering mechanisms.
  arXiv  Detail & Related papers  (2023-02-28T14:14:17Z)
• Fleet-DAgger: Interactive Robot Fleet Learning with Scalable Human Supervision [72.4735163268491]
  Commercial and industrial deployments of robot fleets often fall back on remote human teleoperators during execution. We formalize the Interactive Fleet Learning (IFL) setting, in which multiple robots interactively query and learn from multiple human supervisors. We propose Fleet-DAgger, a family of IFL algorithms, and compare a novel Fleet-DAgger algorithm to 4 baselines in simulation.
  arXiv  Detail & Related papers  (2022-06-29T01:23:57Z)
• SABER: Data-Driven Motion Planner for Autonomously Navigating Heterogeneous Robots [112.2491765424719]
  We present an end-to-end online motion planning framework that uses a data-driven approach to navigate a heterogeneous robot team towards a global goal. We use model predictive control (SMPC) to calculate control inputs that satisfy robot dynamics, and consider uncertainty during obstacle avoidance with chance constraints. recurrent neural networks are used to provide a quick estimate of future state uncertainty considered in the SMPC finite-time horizon solution. A Deep Q-learning agent is employed to serve as a high-level path planner, providing the SMPC with target positions that move the robots towards a desired global goal.
  arXiv  Detail & Related papers  (2021-08-03T02:56:21Z)
• Machine Learning-Based Automated Design Space Exploration for Autonomous Aerial Robots [55.056709056795206]
  Building domain-specific architectures for autonomous aerial robots is challenging due to a lack of systematic methodology for designing onboard compute. We introduce a novel performance model called the F-1 roofline to help architects understand how to build a balanced computing system. To navigate the cyber-physical design space automatically, we subsequently introduce AutoPilot.
  arXiv  Detail & Related papers  (2021-02-05T03:50:54Z)
• Embedded Computer Vision System Applied to a Four-Legged Line Follower Robot [0.0]
  This project aims to drive a robot using an automated computer vision embedded system, connecting the robot's vision to its behavior. The robot is applied on a typical mobile robot's issue: line following. Decision making of where to move next is based on the line center of the path and is fully automated.
  arXiv  Detail & Related papers  (2021-01-12T23:52:53Z)
• Peer-Assisted Robotic Learning: A Data-Driven Collaborative Learning Approach for Cloud Robotic Systems [26.01178673629753]
  Peer-Assisted Robotic Learning (PARL) in robotics is inspired by the peer-assisted learning in cognitive psychology and pedagogy. Data and models are shared by robots to the cloud after semantic computing and training locally. Finally, fine tune this larger shared dataset in the cloud to local robots.
  arXiv  Detail & Related papers  (2020-10-16T10:52:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.