Factorized Gaussian Process Variational Autoencoders

• URL: http://arxiv.org/abs/2011.07255v1
• Date: Sat, 14 Nov 2020 10:24:10 GMT
• Title: Factorized Gaussian Process Variational Autoencoders
• Authors: Metod Jazbec, Michael Pearce, Vincent Fortuin
• Abstract summary: Variational autoencoders often assume isotropic Gaussian priors and mean-field posteriors, hence do not exploit structure in scenarios where we may expect similarity or consistency across latent variables. We propose a more scalable extension of these models by leveraging the independence of the auxiliary features, which is present in many datasets.
• Score: 6.866104126509981
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Variational autoencoders often assume isotropic Gaussian priors and mean-field posteriors, hence do not exploit structure in scenarios where we may expect similarity or consistency across latent variables. Gaussian process variational autoencoders alleviate this problem through the use of a latent Gaussian process, but lead to a cubic inference time complexity. We propose a more scalable extension of these models by leveraging the independence of the auxiliary features, which is present in many datasets. Our model factorizes the latent kernel across these features in different dimensions, leading to a significant speed-up (in theory and practice), while empirically performing comparably to existing non-scalable approaches. Moreover, our approach allows for additional modeling of global latent information and for more general extrapolation to unseen input combinations.

Related papers

• Fully Bayesian Differential Gaussian Processes through Stochastic Differential Equations [7.439555720106548]
  We propose a fully Bayesian approach that treats the kernel hyper parameters as random variables and constructs coupled differential equations (SDEs) to learn their posterior distribution and that of inducing points. Our approach provides a time-varying, comprehensive, and realistic posterior approximation through coupling variables using SDE methods. Our work opens up exciting research avenues for advancing Bayesian inference and offers a powerful modeling tool for continuous-time Gaussian processes.
  arXiv  Detail & Related papers  (2024-08-12T11:41:07Z)
• Combining additivity and active subspaces for high-dimensional Gaussian process modeling [2.7140711924836816]
  We show how to combine high-dimensional Gaussian process modeling with a multi-fidelity strategy. Our contribution for high-dimensional Gaussian process modeling is to combine them with a multi-fidelity strategy, showcasing the advantages through experiments on synthetic functions and datasets.
  arXiv  Detail & Related papers  (2024-02-06T08:49:27Z)
• Heterogeneous Multi-Task Gaussian Cox Processes [61.67344039414193]
  We present a novel extension of multi-task Gaussian Cox processes for modeling heterogeneous correlated tasks jointly. A MOGP prior over the parameters of the dedicated likelihoods for classification, regression and point process tasks can facilitate sharing of information between heterogeneous tasks. We derive a mean-field approximation to realize closed-form iterative updates for estimating model parameters.
  arXiv  Detail & Related papers  (2023-08-29T15:01:01Z)
• VTAE: Variational Transformer Autoencoder with Manifolds Learning [144.0546653941249]
  Deep generative models have demonstrated successful applications in learning non-linear data distributions through a number of latent variables. The nonlinearity of the generator implies that the latent space shows an unsatisfactory projection of the data space, which results in poor representation learning. We show that geodesics and accurate computation can substantially improve the performance of deep generative models.
  arXiv  Detail & Related papers  (2023-04-03T13:13:19Z)
• Linear Self-Attention Approximation via Trainable Feedforward Kernel [77.34726150561087]
  In pursuit of faster computation, Efficient Transformers demonstrate an impressive variety of approaches. We aim to expand the idea of trainable kernel methods to approximate the self-attention mechanism of the Transformer architecture.
  arXiv  Detail & Related papers  (2022-11-08T08:14:11Z)
• A Variational Inference Approach to Inverse Problems with Gamma Hyperpriors [60.489902135153415]
  This paper introduces a variational iterative alternating scheme for hierarchical inverse problems with gamma hyperpriors. The proposed variational inference approach yields accurate reconstruction, provides meaningful uncertainty quantification, and is easy to implement.
  arXiv  Detail & Related papers  (2021-11-26T06:33:29Z)
• Scalable Variational Gaussian Processes via Harmonic Kernel Decomposition [54.07797071198249]
  We introduce a new scalable variational Gaussian process approximation which provides a high fidelity approximation while retaining general applicability. We demonstrate that, on a range of regression and classification problems, our approach can exploit input space symmetries such as translations and reflections. Notably, our approach achieves state-of-the-art results on CIFAR-10 among pure GP models.
  arXiv  Detail & Related papers  (2021-06-10T18:17:57Z)
• Recyclable Gaussian Processes [0.0]
  We present a new framework for recycling independent variational approximations to Gaussian processes. The main contribution is the construction of variational ensembles given a dictionary of fitted Gaussian processes. Our framework allows for regression, classification and heterogeneous tasks.
  arXiv  Detail & Related papers  (2020-10-06T09:01:55Z)
• Scalable Variational Gaussian Process Regression Networks [19.699020509495437]
  We propose a scalable variational inference algorithm for GPRN. We tensorize the output space and introduce tensor/matrix-normal variational posteriors to capture the posterior correlations. We demonstrate the advantages of our method in several real-world applications.
  arXiv  Detail & Related papers  (2020-03-25T16:39:47Z)
• Variational Hyper RNN for Sequence Modeling [69.0659591456772]
  We propose a novel probabilistic sequence model that excels at capturing high variability in time series data. Our method uses temporal latent variables to capture information about the underlying data pattern. The efficacy of the proposed method is demonstrated on a range of synthetic and real-world sequential data.
  arXiv  Detail & Related papers  (2020-02-24T19:30:32Z)
• Randomly Projected Additive Gaussian Processes for Regression [37.367935314532154]
  We use additive sums of kernels for GP regression, where each kernel operates on a different random projection of its inputs. We prove this convergence and its rate, and propose a deterministic approach that converges more quickly than purely random projections.
  arXiv  Detail & Related papers  (2019-12-30T07:26:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.