Deep Variational Implicit Processes
- URL: http://arxiv.org/abs/2206.06720v1
- Date: Tue, 14 Jun 2022 10:04:41 GMT
- Title: Deep Variational Implicit Processes
- Authors: Luis A. Ortega, Sim\'on Rodr\'iguez Santana and Daniel
Hern\'andez-Lobato
- Abstract summary: Implicit processes (IPs) are a generalization of Gaussian processes (GPs)
We propose a multi-layer generalization of IPs called the Deep Variational Implicit process (DVIP)
- Score: 0.0
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Implicit processes (IPs) are a generalization of Gaussian processes (GPs).
IPs may lack a closed-form expression but are easy to sample from. Examples
include, among others, Bayesian neural networks or neural samplers. IPs can be
used as priors over functions, resulting in flexible models with
well-calibrated prediction uncertainty estimates. Methods based on IPs usually
carry out function-space approximate inference, which overcomes some of the
difficulties of parameter-space approximate inference. Nevertheless, the
approximations employed often limit the expressiveness of the final model,
resulting, \emph{e.g.}, in a Gaussian predictive distribution, which can be
restrictive. We propose here a multi-layer generalization of IPs called the
Deep Variational Implicit process (DVIP). This generalization is similar to
that of deep GPs over GPs, but it is more flexible due to the use of IPs as the
prior distribution over the latent functions. We describe a scalable
variational inference algorithm for training DVIP and show that it outperforms
previous IP-based methods and also deep GPs. We support these claims via
extensive regression and classification experiments. We also evaluate DVIP on
large datasets with up to several million data instances to illustrate its good
scalability and performance.
Related papers
- Amortized Variational Inference for Deep Gaussian Processes [0.0]
Deep Gaussian processes (DGPs) are multilayer generalizations of Gaussian processes (GPs)
We introduce amortized variational inference for DGPs, which learns an inference function that maps each observation to variational parameters.
Our method performs similarly or better than previous approaches at less computational cost.
arXiv Detail & Related papers (2024-09-18T20:23:27Z) - FaDIn: Fast Discretized Inference for Hawkes Processes with General
Parametric Kernels [82.53569355337586]
This work offers an efficient solution to temporal point processes inference using general parametric kernels with finite support.
The method's effectiveness is evaluated by modeling the occurrence of stimuli-induced patterns from brain signals recorded with magnetoencephalography (MEG)
Results show that the proposed approach leads to an improved estimation of pattern latency than the state-of-the-art.
arXiv Detail & Related papers (2022-10-10T12:35:02Z) - Scaling Structured Inference with Randomization [64.18063627155128]
We propose a family of dynamic programming (RDP) randomized for scaling structured models to tens of thousands of latent states.
Our method is widely applicable to classical DP-based inference.
It is also compatible with automatic differentiation so can be integrated with neural networks seamlessly.
arXiv Detail & Related papers (2021-12-07T11:26:41Z) - Non-Gaussian Gaussian Processes for Few-Shot Regression [71.33730039795921]
We propose an invertible ODE-based mapping that operates on each component of the random variable vectors and shares the parameters across all of them.
NGGPs outperform the competing state-of-the-art approaches on a diversified set of benchmarks and applications.
arXiv Detail & Related papers (2021-10-26T10:45:25Z) - Sparse Implicit Processes for Approximate Inference [0.0]
Implicit processes (IPs) are flexible priors that can describe models such as Bayesian neural networks, neural samplers and data generators.
We propose a method for simultaneously adjusting the parameters of the prior IP and the approximate posterior IP.
The result is a scalable method for approximate inference with IPs that can tune the prior IP parameters to the data, and that provides accurate non-Gaussian predictive distributions.
arXiv Detail & Related papers (2021-10-14T18:00:04Z) - Incremental Ensemble Gaussian Processes [53.3291389385672]
We propose an incremental ensemble (IE-) GP framework, where an EGP meta-learner employs an it ensemble of GP learners, each having a unique kernel belonging to a prescribed kernel dictionary.
With each GP expert leveraging the random feature-based approximation to perform online prediction and model update with it scalability, the EGP meta-learner capitalizes on data-adaptive weights to synthesize the per-expert predictions.
The novel IE-GP is generalized to accommodate time-varying functions by modeling structured dynamics at the EGP meta-learner and within each GP learner.
arXiv Detail & Related papers (2021-10-13T15:11:25Z) - Predicting Deep Neural Network Generalization with Perturbation Response
Curves [58.8755389068888]
We propose a new framework for evaluating the generalization capabilities of trained networks.
Specifically, we introduce two new measures for accurately predicting generalization gaps.
We attain better predictive scores than the current state-of-the-art measures on a majority of tasks in the Predicting Generalization in Deep Learning (PGDL) NeurIPS 2020 competition.
arXiv Detail & Related papers (2021-06-09T01:37:36Z) - A Tutorial on Sparse Gaussian Processes and Variational Inference [12.827024991969578]
This tutorial provides access to the basic matter for readers without prior knowledge in both GPs and VI.
The framework can naturally handle a wide scope of supervised learning problems, ranging from regression with heteroscedastic and non-Gaussian likelihoods to classification problems with discrete labels, but also multilabel problems.
arXiv Detail & Related papers (2020-12-27T15:25:13Z) - Likelihood-Free Inference with Deep Gaussian Processes [70.74203794847344]
Surrogate models have been successfully used in likelihood-free inference to decrease the number of simulator evaluations.
We propose a Deep Gaussian Process (DGP) surrogate model that can handle more irregularly behaved target distributions.
Our experiments show how DGPs can outperform GPs on objective functions with multimodal distributions and maintain a comparable performance in unimodal cases.
arXiv Detail & Related papers (2020-06-18T14:24:05Z) - Deep Sigma Point Processes [22.5396672566053]
We introduce a class of parametric models inspired by the compositional structure of Deep Gaussian Processes (DGPs)
Deep Sigma Point Processes (DSPPs) retain many of the attractive features of (variational) DGPs, including mini-batch training and predictive uncertainty that is controlled by kernel basis functions.
arXiv Detail & Related papers (2020-02-21T03:40:35Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.