Scalable Perception-Action-Communication Loops with Convolutional and Graph Neural Networks

• URL: http://arxiv.org/abs/2106.13358v1
• Date: Thu, 24 Jun 2021 23:57:21 GMT
• Title: Scalable Perception-Action-Communication Loops with Convolutional and Graph Neural Networks
• Authors: Ting-Kuei Hu, Fernando Gama, Tianlong Chen, Wenqing Zheng, Zhangyang Wang, Alejandro Ribeiro, Brian M. Sadler
• Abstract summary: We present a perception-action-communication loop design using Vision-based Graph Aggregation and Inference (VGAI) Our framework is implemented by a cascade of a convolutional and a graph neural network (CNN / GNN), addressing agent-level visual perception and feature learning. We demonstrate that VGAI yields performance comparable to or better than other decentralized controllers.
• Score: 208.15591625749272
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we present a perception-action-communication loop design using Vision-based Graph Aggregation and Inference (VGAI). This multi-agent decentralized learning-to-control framework maps raw visual observations to agent actions, aided by local communication among neighboring agents. Our framework is implemented by a cascade of a convolutional and a graph neural network (CNN / GNN), addressing agent-level visual perception and feature learning, as well as swarm-level communication, local information aggregation and agent action inference, respectively. By jointly training the CNN and GNN, image features and communication messages are learned in conjunction to better address the specific task. We use imitation learning to train the VGAI controller in an offline phase, relying on a centralized expert controller. This results in a learned VGAI controller that can be deployed in a distributed manner for online execution. Additionally, the controller exhibits good scaling properties, with training in smaller teams and application in larger teams. Through a multi-agent flocking application, we demonstrate that VGAI yields performance comparable to or better than other decentralized controllers, using only the visual input modality and without accessing precise location or motion state information.

Related papers

• ABC: Aggregation before Communication, a Communication Reduction Framework for Distributed Graph Neural Network Training and Effective Partition [0.0]
  Graph Neural Networks (GNNs) are neural models tailored for graph-structure data and have shown superior performance in learning representations for graph-structured data. In this paper, we study the communication complexity during distributed GNNs training. We show that the new partition paradigm is particularly ideal in the case of dynamic graphs where it is infeasible to control the edge placement due to the unknown of the graph-changing process.
  arXiv  Detail & Related papers  (2022-12-11T04:54:01Z)
• Graph Neural Networks for Multi-Robot Active Information Acquisition [15.900385823366117]
  A team of mobile robots, communicating through an underlying graph, estimates a hidden state expressing a phenomenon of interest. Existing approaches are either not scalable, unable to handle dynamic phenomena or not robust to changes in the communication graph. We propose an Information-aware Graph Block Network (I-GBNet) that aggregates information over the graph representation and provides sequential-decision making in a distributed manner.
  arXiv  Detail & Related papers  (2022-09-24T21:45:06Z)
• Dynamic Graph Message Passing Networks for Visual Recognition [112.49513303433606]
  Modelling long-range dependencies is critical for scene understanding tasks in computer vision. A fully-connected graph is beneficial for such modelling, but its computational overhead is prohibitive. We propose a dynamic graph message passing network, that significantly reduces the computational complexity.
  arXiv  Detail & Related papers  (2022-09-20T14:41:37Z)
• Transferring ConvNet Features from Passive to Active Robot Self-Localization: The Use of Ego-Centric and World-Centric Views [2.362412515574206]
  A standard VPR subsystem is assumed to be available, and its domain-invariant state recognition ability is proposed to be transferred to train the domain-invariant NBV planner. We divide the visual cues that are available from the CNN model into two types: the output layer cue (OLC) and intermediate layer cue (ILC) In our framework, the ILC and OLC are mapped to a state vector and subsequently used to train a multiview NBV planner via deep reinforcement learning.
  arXiv  Detail & Related papers  (2022-04-22T04:42:33Z)
• Wireless Link Scheduling via Graph Representation Learning: A Comparative Study of Different Supervision Levels [4.264192013842096]
  We consider the problem of binary power control, or link scheduling, in wireless interference networks, where the power control policy is trained using graph representation learning. We show how the node embeddings can be trained in several ways, including via supervised, unsupervised, and self-supervised learning.
  arXiv  Detail & Related papers  (2021-10-04T21:22:12Z)
• Learning Connectivity for Data Distribution in Robot Teams [96.39864514115136]
  We propose a task-agnostic, decentralized, low-latency method for data distribution in ad-hoc networks using Graph Neural Networks (GNN) Our approach enables multi-agent algorithms based on global state information to function by ensuring it is available at each robot. We train the distributed GNN communication policies via reinforcement learning using the average Age of Information as the reward function and show that it improves training stability compared to task-specific reward functions.
  arXiv  Detail & Related papers  (2021-03-08T21:48:55Z)
• Decentralized Control with Graph Neural Networks [147.84766857793247]
  We propose a novel framework using graph neural networks (GNNs) to learn decentralized controllers. GNNs are well-suited for the task since they are naturally distributed architectures and exhibit good scalability and transferability properties. The problems of flocking and multi-agent path planning are explored to illustrate the potential of GNNs in learning decentralized controllers.
  arXiv  Detail & Related papers  (2020-12-29T18:59:14Z)
• Graph Neural Networks for Decentralized Controllers [171.6642679604005]
  Dynamical systems comprised of autonomous agents arise in many relevant problems such as robotics, smart grids, or smart cities. Optimal centralized controllers are readily available but face limitations in terms of scalability and practical implementation. We propose a framework using graph neural networks (GNNs) to learn decentralized controllers from data.
  arXiv  Detail & Related papers  (2020-03-23T13:51:18Z)
• VGAI: End-to-End Learning of Vision-Based Decentralized Controllers for Robot Swarms [237.25930757584047]
  We propose to learn decentralized controllers based on solely raw visual inputs. For the first time, that integrates the learning of two key components: communication and visual perception. Our proposed learning framework combines a convolutional neural network (CNN) for each robot to extract messages from the visual inputs, and a graph neural network (GNN) over the entire swarm to transmit, receive and process these messages.
  arXiv  Detail & Related papers  (2020-02-06T15:25:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.