The Gaussian Neural Process

• URL: http://arxiv.org/abs/2101.03606v1
• Date: Sun, 10 Jan 2021 19:15:27 GMT
• Title: The Gaussian Neural Process
• Authors: Wessel P. Bruinsma and James Requeima and Andrew Y. K. Foong and Jonathan Gordon and Richard E. Turner
• Abstract summary: We provide a rigorous analysis of the standard maximum-likelihood objective used to train conditional NPs. We propose a new member to the Neural Process family called the Neural Process (GNP), which models predictive correlations, incorporates translation, provides universal approximation guarantees, and demonstrates encouraging performance.
• Score: 39.81327564209865
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Neural Processes (NPs; Garnelo et al., 2018a,b) are a rich class of models for meta-learning that map data sets directly to predictive stochastic processes. We provide a rigorous analysis of the standard maximum-likelihood objective used to train conditional NPs. Moreover, we propose a new member to the Neural Process family called the Gaussian Neural Process (GNP), which models predictive correlations, incorporates translation equivariance, provides universal approximation guarantees, and demonstrates encouraging performance.

Related papers

• Spectral Convolutional Conditional Neural Processes [4.52069311861025]
  Conditional Neural Processes (CNPs) constitute a family of probabilistic models that harness the flexibility of neural networks to parameterize processes. We propose Spectral Convolutional Conditional Neural Processes (SConvCNPs), a new addition to the NPs family that allows for more efficient representation of functions in the frequency domain.
  arXiv  Detail & Related papers  (2024-04-19T21:13:18Z)
• Vecchia Gaussian Process Ensembles on Internal Representations of Deep Neural Networks [0.0]
  For regression tasks, standard Gaussian processes (GPs) provide natural uncertainty quantification, while deep neural networks (DNNs) excel at representation learning. We propose to combine these two approaches in a hybrid method consisting of an ensemble of GPs built on the output of hidden layers of a DNN.
  arXiv  Detail & Related papers  (2023-05-26T16:19:26Z)
• MARS: Meta-Learning as Score Matching in the Function Space [79.73213540203389]
  We present a novel approach to extracting inductive biases from a set of related datasets. We use functional Bayesian neural network inference, which views the prior as a process and performs inference in the function space. Our approach can seamlessly acquire and represent complex prior knowledge by metalearning the score function of the data-generating process.
  arXiv  Detail & Related papers  (2022-10-24T15:14:26Z)
• Transformer Neural Processes: Uncertainty-Aware Meta Learning Via Sequence Modeling [26.377099481072992]
  We propose Transformer Neural Processes (TNPs) for uncertainty-aware meta learning. We learn TNPs via an autoregressive likelihood-based objective and instantiate it with a novel transformer-based architecture. We show that TNPs achieve state-of-the-art performance on various benchmark problems.
  arXiv  Detail & Related papers  (2022-07-09T02:28:58Z)
• Neural Diffusion Processes [12.744250155946503]
  We propose Neural Diffusion Processes (NDPs), a novel approach that learns to sample from a rich distribution over functions through its finite marginals. We empirically show that NDPs can capture functional distributions close to the true Bayesian posterior. NDPs enable a variety of downstream tasks, including regression, implicit hyper marginalisation, non-Gaussian posterior prediction and global optimisation.
  arXiv  Detail & Related papers  (2022-06-08T16:13:04Z)
• Incremental Ensemble Gaussian Processes [53.3291389385672]
  We propose an incremental ensemble (IE-) GP framework, where an EGP meta-learner employs an it ensemble of GP learners, each having a unique kernel belonging to a prescribed kernel dictionary. With each GP expert leveraging the random feature-based approximation to perform online prediction and model update with it scalability, the EGP meta-learner capitalizes on data-adaptive weights to synthesize the per-expert predictions. The novel IE-GP is generalized to accommodate time-varying functions by modeling structured dynamics at the EGP meta-learner and within each GP learner.
  arXiv  Detail & Related papers  (2021-10-13T15:11:25Z)
• Improving predictions of Bayesian neural nets via local linearization [79.21517734364093]
  We argue that the Gauss-Newton approximation should be understood as a local linearization of the underlying Bayesian neural network (BNN) Because we use this linearized model for posterior inference, we should also predict using this modified model instead of the original one. We refer to this modified predictive as "GLM predictive" and show that it effectively resolves common underfitting problems of the Laplace approximation.
  arXiv  Detail & Related papers  (2020-08-19T12:35:55Z)
• Bootstrapping Neural Processes [114.97111530885093]
  Neural Processes (NPs) implicitly define a broad class of processes with neural networks. NPs still rely on an assumption that uncertainty in processes is modeled by a single latent variable. We propose the Boostrapping Neural Process (BNP), a novel extension of the NP family using the bootstrap.
  arXiv  Detail & Related papers  (2020-08-07T02:23:34Z)
• Meta-Learning Stationary Stochastic Process Prediction with Convolutional Neural Processes [32.02612871707347]
  We propose ConvNP, which endows Neural Processes (NPs) with translation equivariance and extends convolutional conditional NPs to allow for dependencies in the predictive distribution. We demonstrate the strong performance and generalization capabilities of ConvNPs on 1D, regression image completion, and various tasks with real-world-temporal data.
  arXiv  Detail & Related papers  (2020-07-02T18:25:27Z)
• SLEIPNIR: Deterministic and Provably Accurate Feature Expansion for Gaussian Process Regression with Derivatives [86.01677297601624]
  We propose a novel approach for scaling GP regression with derivatives based on quadrature Fourier features. We prove deterministic, non-asymptotic and exponentially fast decaying error bounds which apply for both the approximated kernel as well as the approximated posterior.
  arXiv  Detail & Related papers  (2020-03-05T14:33:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.