Graph Based Gaussian Processes on Restricted Domains

• URL: http://arxiv.org/abs/2010.07242v3
• Date: Tue, 2 Nov 2021 19:51:26 GMT
• Title: Graph Based Gaussian Processes on Restricted Domains
• Authors: David B Dunson, Hau-Tieng Wu and Nan Wu
• Abstract summary: In nonparametric regression, it is common for the inputs to fall in a restricted subset of Euclidean space. We propose a new class of Graph Laplacian based GPs (GL-GPs) which learn a covariance that respects the geometry of the input domain. We provide substantial theoretical support for the GL-GP methodology, and illustrate performance gains in various applications.
• Score: 13.416168979487118
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In nonparametric regression, it is common for the inputs to fall in a restricted subset of Euclidean space. Typical kernel-based methods that do not take into account the intrinsic geometry of the domain across which observations are collected may produce sub-optimal results. In this article, we focus on solving this problem in the context of Gaussian process (GP) models, proposing a new class of Graph Laplacian based GPs (GL-GPs), which learn a covariance that respects the geometry of the input domain. As the heat kernel is intractable computationally, we approximate the covariance using finitely-many eigenpairs of the Graph Laplacian (GL). The GL is constructed from a kernel which depends only on the Euclidean coordinates of the inputs. Hence, we can benefit from the full knowledge about the kernel to extend the covariance structure to newly arriving samples by a Nystr\"{o}m type extension. We provide substantial theoretical support for the GL-GP methodology, and illustrate performance gains in various applications.

Related papers

• Taming graph kernels with random features [17.482280753348288]
  We introduce the mechanism of graph random features (GRFs) GRFs can be used to construct unbiased randomized estimators of several important kernels defined on graphs' nodes.
  arXiv  Detail & Related papers  (2023-04-29T03:04:49Z)
• Graph Neural Network-Inspired Kernels for Gaussian Processes in Semi-Supervised Learning [4.644263115284322]
  Graph neural networks (GNNs) emerged recently as a promising class of models for graph-structured data in semi-supervised learning. We introduce this inductive bias into GPs to improve their predictive performance for graph-structured data. We show that these graph-based kernels lead to competitive classification and regression performance, as well as advantages in time, compared with the respective GNNs.
  arXiv  Detail & Related papers  (2023-02-12T01:07:56Z)
• A mixed-categorical correlation kernel for Gaussian process [0.0]
  We present a kernel-based approach that extends continuous exponential kernels to handle mixed-categorical variables. The proposed kernel leads to a new GP surrogate that generalizes both the continuous relaxation and the Gower distance based GP models.
  arXiv  Detail & Related papers  (2022-11-15T16:13:04Z)
• Shallow and Deep Nonparametric Convolutions for Gaussian Processes [0.0]
  We introduce a nonparametric process convolution formulation for GPs that alleviates weaknesses by using a functional sampling approach. We propose a composition of these nonparametric convolutions that serves as an alternative to classic deep GP models.
  arXiv  Detail & Related papers  (2022-06-17T19:03:04Z)
• Exact Gaussian Processes for Massive Datasets via Non-Stationary Sparsity-Discovering Kernels [0.0]
  We propose to take advantage of naturally-structured sparsity by allowing the kernel to discover -- instead of induce -- sparse structure. The principle of ultra-flexible, compactly-supported, and non-stationary kernels, combined with HPC and constrained optimization, lets us scale exact GPs well beyond 5 million data points.
  arXiv  Detail & Related papers  (2022-05-18T16:56:53Z)
• Non-Gaussian Gaussian Processes for Few-Shot Regression [71.33730039795921]
  We propose an invertible ODE-based mapping that operates on each component of the random variable vectors and shares the parameters across all of them. NGGPs outperform the competing state-of-the-art approaches on a diversified set of benchmarks and applications.
  arXiv  Detail & Related papers  (2021-10-26T10:45:25Z)
• Scalable Variational Gaussian Processes via Harmonic Kernel Decomposition [54.07797071198249]
  We introduce a new scalable variational Gaussian process approximation which provides a high fidelity approximation while retaining general applicability. We demonstrate that, on a range of regression and classification problems, our approach can exploit input space symmetries such as translations and reflections. Notably, our approach achieves state-of-the-art results on CIFAR-10 among pure GP models.
  arXiv  Detail & Related papers  (2021-06-10T18:17:57Z)
• Learning Domain-invariant Graph for Adaptive Semi-supervised Domain Adaptation with Few Labeled Source Samples [65.55521019202557]
  Domain adaptation aims to generalize a model from a source domain to tackle tasks in a related but different target domain. Traditional domain adaptation algorithms assume that enough labeled data, which are treated as the prior knowledge are available in the source domain. We propose a Domain-invariant Graph Learning (DGL) approach for domain adaptation with only a few labeled source samples.
  arXiv  Detail & Related papers  (2020-08-21T08:13:25Z)
• SLEIPNIR: Deterministic and Provably Accurate Feature Expansion for Gaussian Process Regression with Derivatives [86.01677297601624]
  We propose a novel approach for scaling GP regression with derivatives based on quadrature Fourier features. We prove deterministic, non-asymptotic and exponentially fast decaying error bounds which apply for both the approximated kernel as well as the approximated posterior.
  arXiv  Detail & Related papers  (2020-03-05T14:33:20Z)
• Infinitely Wide Graph Convolutional Networks: Semi-supervised Learning via Gaussian Processes [144.6048446370369]
  Graph convolutional neural networks(GCNs) have recently demonstrated promising results on graph-based semi-supervised classification. We propose a GP regression model via GCNs(GPGC) for graph-based semi-supervised learning. We conduct extensive experiments to evaluate GPGC and demonstrate that it outperforms other state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-02-26T10:02:32Z)
• Embedding Graph Auto-Encoder for Graph Clustering [90.8576971748142]
  Graph auto-encoder (GAE) models are based on semi-supervised graph convolution networks (GCN) We design a specific GAE-based model for graph clustering to be consistent with the theory, namely Embedding Graph Auto-Encoder (EGAE) EGAE consists of one encoder and dual decoders.
  arXiv  Detail & Related papers  (2020-02-20T09:53:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.