Grid-to-Graph: Flexible Spatial Relational Inductive Biases for
Reinforcement Learning
- URL: http://arxiv.org/abs/2102.04220v1
- Date: Mon, 8 Feb 2021 14:15:13 GMT
- Title: Grid-to-Graph: Flexible Spatial Relational Inductive Biases for
Reinforcement Learning
- Authors: Zhengyao Jiang, Pasquale Minervini, Minqi Jiang, Tim Rocktaschel
- Abstract summary: We show that we can incorporate relational inductive biases, encoded in the form of relational graphs, into agents.
We propose Grid-to-Graph (GTG), a mapping from grid structures to relational graphs that carry useful inductive biases.
We show that GTG produces agents that can jointly reason over observations and environment encoded dynamics in knowledge bases.
- Score: 8.169818701603313
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Although reinforcement learning has been successfully applied in many domains
in recent years, we still lack agents that can systematically generalize. While
relational inductive biases that fit a task can improve generalization of RL
agents, these biases are commonly hard-coded directly in the agent's neural
architecture. In this work, we show that we can incorporate relational
inductive biases, encoded in the form of relational graphs, into agents. Based
on this insight, we propose Grid-to-Graph (GTG), a mapping from grid structures
to relational graphs that carry useful spatial relational inductive biases when
processed through a Relational Graph Convolution Network (R-GCN). We show that,
with GTG, R-GCNs generalize better both in terms of in-distribution and
out-of-distribution compared to baselines based on Convolutional Neural
Networks and Neural Logic Machines on challenging procedurally generated
environments and MinAtar. Furthermore, we show that GTG produces agents that
can jointly reason over observations and environment dynamics encoded in
knowledge bases.
Related papers
- A Manifold Perspective on the Statistical Generalization of Graph Neural Networks [84.01980526069075]
Graph Neural Networks (GNNs) combine information from adjacent nodes by successive applications of graph convolutions.
We study the generalization gaps of GNNs on both node-level and graph-level tasks.
We show that the generalization gaps decrease with the number of nodes in the training graphs.
arXiv Detail & Related papers (2024-06-07T19:25:02Z) - Advective Diffusion Transformers for Topological Generalization in Graph
Learning [69.2894350228753]
We show how graph diffusion equations extrapolate and generalize in the presence of varying graph topologies.
We propose a novel graph encoder backbone, Advective Diffusion Transformer (ADiT), inspired by advective graph diffusion equations.
arXiv Detail & Related papers (2023-10-10T08:40:47Z) - Graph Neural Network-Inspired Kernels for Gaussian Processes in
Semi-Supervised Learning [4.644263115284322]
Graph neural networks (GNNs) emerged recently as a promising class of models for graph-structured data in semi-supervised learning.
We introduce this inductive bias into GPs to improve their predictive performance for graph-structured data.
We show that these graph-based kernels lead to competitive classification and regression performance, as well as advantages in time, compared with the respective GNNs.
arXiv Detail & Related papers (2023-02-12T01:07:56Z) - Beyond Graph Convolutional Network: An Interpretable
Regularizer-centered Optimization Framework [12.116373546916078]
Graph convolutional networks (GCNs) have been attracting widespread attentions due to their encouraging performance and powerful generalizations.
In this paper, we induce an interpretable regularizer-centerd optimization framework, in which by building appropriate regularizers we can interpret most GCNs.
Under the proposed framework, we devise a dual-regularizer graph convolutional network (dubbed tsGCN) to capture topological and semantic structures from graph data.
arXiv Detail & Related papers (2023-01-11T05:51:33Z) - Stable and Transferable Hyper-Graph Neural Networks [95.07035704188984]
We introduce an architecture for processing signals supported on hypergraphs via graph neural networks (GNNs)
We provide a framework for bounding the stability and transferability error of GNNs across arbitrary graphs via spectral similarity.
arXiv Detail & Related papers (2022-11-11T23:44:20Z) - Relation Embedding based Graph Neural Networks for Handling
Heterogeneous Graph [58.99478502486377]
We propose a simple yet efficient framework to make the homogeneous GNNs have adequate ability to handle heterogeneous graphs.
Specifically, we propose Relation Embedding based Graph Neural Networks (RE-GNNs), which employ only one parameter per relation to embed the importance of edge type relations and self-loop connections.
arXiv Detail & Related papers (2022-09-23T05:24:18Z) - Complex-Value Spatio-temporal Graph Convolutional Neural Networks and
its Applications to Electric Power Systems AI [24.914412344973996]
We generalize graph convolutional neural networks (GCN) to the complex domain.
We prove that complex-valued GCNs are stable with respect to perturbations of the underlying graph support.
We apply complex GCN to power grid state forecasting, power grid-attack detection and localization.
arXiv Detail & Related papers (2022-08-17T18:56:48Z) - Learning Graph Structure from Convolutional Mixtures [119.45320143101381]
We propose a graph convolutional relationship between the observed and latent graphs, and formulate the graph learning task as a network inverse (deconvolution) problem.
In lieu of eigendecomposition-based spectral methods, we unroll and truncate proximal gradient iterations to arrive at a parameterized neural network architecture that we call a Graph Deconvolution Network (GDN)
GDNs can learn a distribution of graphs in a supervised fashion, perform link prediction or edge-weight regression tasks by adapting the loss function, and they are inherently inductive.
arXiv Detail & Related papers (2022-05-19T14:08:15Z) - Learning Connectivity with Graph Convolutional Networks for
Skeleton-based Action Recognition [14.924672048447338]
We introduce a novel framework for graph convolutional networks that learns the topological properties of graphs.
The design principle of our method is based on the optimization of a constrained objective function.
Experiments conducted on the challenging task of skeleton-based action recognition shows the superiority of the proposed method.
arXiv Detail & Related papers (2021-12-06T19:43:26Z) - Infinitely Wide Graph Convolutional Networks: Semi-supervised Learning
via Gaussian Processes [144.6048446370369]
Graph convolutional neural networks(GCNs) have recently demonstrated promising results on graph-based semi-supervised classification.
We propose a GP regression model via GCNs(GPGC) for graph-based semi-supervised learning.
We conduct extensive experiments to evaluate GPGC and demonstrate that it outperforms other state-of-the-art methods.
arXiv Detail & Related papers (2020-02-26T10:02:32Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.