Convolutional Normalizing Flows for Deep Gaussian Processes
- URL: http://arxiv.org/abs/2104.08472v1
- Date: Sat, 17 Apr 2021 07:25:25 GMT
- Title: Convolutional Normalizing Flows for Deep Gaussian Processes
- Authors: Haibin Yu, Bryan Kian Hsiang Low, Patrick Jaillet, Dapeng Liu
- Abstract summary: This paper introduces a new approach for specifying flexible, arbitrarily complex, and scalable approximate posterior distributions.
A novel convolutional normalizing flow (CNF) is developed to improve the time efficiency and capture dependency between layers.
Empirical evaluation demonstrates that CNF DGP outperforms the state-of-the-art approximation methods for DGPs.
- Score: 40.10797051603641
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Deep Gaussian processes (DGPs), a hierarchical composition of GP models, have
successfully boosted the expressive power than the single-layer counterpart.
However, it is impossible to perform exact inference in DGPs, which has
motivated the recent development of variational inference based methods.
Unfortunately, these methods either yield a biased posterior belief or are
difficult to evaluate the convergence. This paper, on the contrary, introduces
a new approach for specifying flexible, arbitrarily complex, and scalable
approximate posterior distributions. The posterior distribution is constructed
through a normalizing flow (NF) which transforms a simple initial probability
into a more complex one through a sequence of invertible transformations.
Moreover, a novel convolutional normalizing flow (CNF) is developed to improve
the time efficiency and capture dependency between layers. Empirical evaluation
demonstrates that CNF DGP outperforms the state-of-the-art approximation
methods for DGPs.
Related papers
- Amortized Variational Inference for Deep Gaussian Processes [0.0]
Deep Gaussian processes (DGPs) are multilayer generalizations of Gaussian processes (GPs)
We introduce amortized variational inference for DGPs, which learns an inference function that maps each observation to variational parameters.
Our method performs similarly or better than previous approaches at less computational cost.
arXiv Detail & Related papers (2024-09-18T20:23:27Z) - Deep Transformed Gaussian Processes [0.0]
Transformed Gaussian Processes (TGPs) are processes specified by transforming samples from the joint distribution from a prior process (typically a GP) using an invertible transformation.
We propose a generalization of TGPs named Deep Transformed Gaussian Processes (DTGPs), which follows the trend of concatenating layers of processes.
Experiments conducted evaluate the proposed DTGPs in multiple regression datasets, achieving good scalability and performance.
arXiv Detail & Related papers (2023-10-27T16:09:39Z) - Matching Normalizing Flows and Probability Paths on Manifolds [57.95251557443005]
Continuous Normalizing Flows (CNFs) are generative models that transform a prior distribution to a model distribution by solving an ordinary differential equation (ODE)
We propose to train CNFs by minimizing probability path divergence (PPD), a novel family of divergences between the probability density path generated by the CNF and a target probability density path.
We show that CNFs learned by minimizing PPD achieve state-of-the-art results in likelihoods and sample quality on existing low-dimensional manifold benchmarks.
arXiv Detail & Related papers (2022-07-11T08:50:19Z) - Shallow and Deep Nonparametric Convolutions for Gaussian Processes [0.0]
We introduce a nonparametric process convolution formulation for GPs that alleviates weaknesses by using a functional sampling approach.
We propose a composition of these nonparametric convolutions that serves as an alternative to classic deep GP models.
arXiv Detail & Related papers (2022-06-17T19:03:04Z) - Non-Gaussian Gaussian Processes for Few-Shot Regression [71.33730039795921]
We propose an invertible ODE-based mapping that operates on each component of the random variable vectors and shares the parameters across all of them.
NGGPs outperform the competing state-of-the-art approaches on a diversified set of benchmarks and applications.
arXiv Detail & Related papers (2021-10-26T10:45:25Z) - Scalable Variational Gaussian Processes via Harmonic Kernel
Decomposition [54.07797071198249]
We introduce a new scalable variational Gaussian process approximation which provides a high fidelity approximation while retaining general applicability.
We demonstrate that, on a range of regression and classification problems, our approach can exploit input space symmetries such as translations and reflections.
Notably, our approach achieves state-of-the-art results on CIFAR-10 among pure GP models.
arXiv Detail & Related papers (2021-06-10T18:17:57Z) - Likelihood-Free Inference with Deep Gaussian Processes [70.74203794847344]
Surrogate models have been successfully used in likelihood-free inference to decrease the number of simulator evaluations.
We propose a Deep Gaussian Process (DGP) surrogate model that can handle more irregularly behaved target distributions.
Our experiments show how DGPs can outperform GPs on objective functions with multimodal distributions and maintain a comparable performance in unimodal cases.
arXiv Detail & Related papers (2020-06-18T14:24:05Z) - Conditional Deep Gaussian Processes: multi-fidelity kernel learning [6.599344783327053]
We propose the conditional DGP model in which the latent GPs are directly supported by the fixed lower fidelity data.
Experiments with synthetic and high dimensional data show comparable performance against other multi-fidelity regression methods.
We conclude that, with the low fidelity data and the hierarchical DGP structure, the effective kernel encodes the inductive bias for true function.
arXiv Detail & Related papers (2020-02-07T14:56:11Z) - Learning Likelihoods with Conditional Normalizing Flows [54.60456010771409]
Conditional normalizing flows (CNFs) are efficient in sampling and inference.
We present a study of CNFs where the base density to output space mapping is conditioned on an input x, to model conditional densities p(y|x)
arXiv Detail & Related papers (2019-11-29T19:17:58Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.