Reward Propagation Using Graph Convolutional Networks
- URL: http://arxiv.org/abs/2010.02474v2
- Date: Mon, 2 Nov 2020 02:37:16 GMT
- Title: Reward Propagation Using Graph Convolutional Networks
- Authors: Martin Klissarov and Doina Precup
- Abstract summary: We propose a new framework for learning potential functions by leveraging ideas from graph representation learning.
Our approach relies on Graph Convolutional Networks which we use as a key ingredient in combination with the probabilistic inference view of reinforcement learning.
- Score: 61.32891095232801
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Potential-based reward shaping provides an approach for designing good reward
functions, with the purpose of speeding up learning. However, automatically
finding potential functions for complex environments is a difficult problem (in
fact, of the same difficulty as learning a value function from scratch). We
propose a new framework for learning potential functions by leveraging ideas
from graph representation learning. Our approach relies on Graph Convolutional
Networks which we use as a key ingredient in combination with the probabilistic
inference view of reinforcement learning. More precisely, we leverage Graph
Convolutional Networks to perform message passing from rewarding states. The
propagated messages can then be used as potential functions for reward shaping
to accelerate learning. We verify empirically that our approach can achieve
considerable improvements in both small and high-dimensional control problems.
Related papers
- Joint Feature and Differentiable $ k $-NN Graph Learning using Dirichlet
Energy [103.74640329539389]
We propose a deep FS method that simultaneously conducts feature selection and differentiable $ k $-NN graph learning.
We employ Optimal Transport theory to address the non-differentiability issue of learning $ k $-NN graphs in neural networks.
We validate the effectiveness of our model with extensive experiments on both synthetic and real-world datasets.
arXiv Detail & Related papers (2023-05-21T08:15:55Z) - Generative Flow Networks for Precise Reward-Oriented Active Learning on
Graphs [34.76241250013461]
We formulate the graph active learning problem as a generative process, named GFlowGNN, which generates various samples through sequential actions.
We show that the proposed approach has good exploration capability and transferability, outperforming various state-of-the-art methods.
arXiv Detail & Related papers (2023-04-24T10:47:08Z) - GIF: A General Graph Unlearning Strategy via Influence Function [63.52038638220563]
Graph Influence Function (GIF) is a model-agnostic unlearning method that can efficiently and accurately estimate parameter changes in response to a $epsilon$-mass perturbation in deleted data.
We conduct extensive experiments on four representative GNN models and three benchmark datasets to justify GIF's superiority in terms of unlearning efficacy, model utility, and unlearning efficiency.
arXiv Detail & Related papers (2023-04-06T03:02:54Z) - Learnable Graph Convolutional Network and Feature Fusion for Multi-view
Learning [30.74535386745822]
This paper proposes a joint deep learning framework called Learnable Graph Convolutional Network and Feature Fusion (LGCN-FF)
It consists of two stages: feature fusion network and learnable graph convolutional network.
The proposed LGCN-FF is validated to be superior to various state-of-the-art methods in multi-view semi-supervised classification.
arXiv Detail & Related papers (2022-11-16T19:07:12Z) - Inducing Gaussian Process Networks [80.40892394020797]
We propose inducing Gaussian process networks (IGN), a simple framework for simultaneously learning the feature space as well as the inducing points.
The inducing points, in particular, are learned directly in the feature space, enabling a seamless representation of complex structured domains.
We report on experimental results for real-world data sets showing that IGNs provide significant advances over state-of-the-art methods.
arXiv Detail & Related papers (2022-04-21T05:27:09Z) - Potential-based Reward Shaping in Sokoban [5.563631490799427]
We study whether we can use a search algorithm(A*) to automatically generate a potential function for reward shaping in Sokoban.
Results showed that learning with shaped reward function is faster than learning from scratch.
Results indicate that distance functions could be a suitable function for Sokoban.
arXiv Detail & Related papers (2021-09-10T06:28:09Z) - xRAI: Explainable Representations through AI [10.345196226375455]
We present an approach for extracting symbolic representations of the mathematical function a neural network was supposed to learn from the trained network.
The approach is based on the idea of training a so-called interpretation network that receives the weights and biases of the trained network as input and outputs the numerical representation of the function the network was supposed to learn that can be directly translated into a symbolic representation.
arXiv Detail & Related papers (2020-12-10T22:49:29Z) - Graph-Based Neural Network Models with Multiple Self-Supervised
Auxiliary Tasks [79.28094304325116]
Graph Convolutional Networks are among the most promising approaches for capturing relationships among structured data points.
We propose three novel self-supervised auxiliary tasks to train graph-based neural network models in a multi-task fashion.
arXiv Detail & Related papers (2020-11-14T11:09:51Z) - Geometrically Principled Connections in Graph Neural Networks [66.51286736506658]
We argue geometry should remain the primary driving force behind innovation in the emerging field of geometric deep learning.
We relate graph neural networks to widely successful computer graphics and data approximation models: radial basis functions (RBFs)
We introduce affine skip connections, a novel building block formed by combining a fully connected layer with any graph convolution operator.
arXiv Detail & Related papers (2020-04-06T13:25:46Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.