Efficient Gaussian Neural Processes for Regression

• URL: http://arxiv.org/abs/2108.09676v2
• Date: Tue, 24 Aug 2021 08:58:58 GMT
• Title: Efficient Gaussian Neural Processes for Regression
• Authors: Stratis Markou, James Requeima, Wessel Bruinsma, Richard Turner
• Abstract summary: Conditional Neural Processes (CNPs) produce well-calibrated predictions, enable fast inference at test time, and are trainable via a simple maximum likelihood procedure. A limitation of CNPs is their inability to model dependencies in the outputs. We present an alternative way to model output dependencies which also lends itself maximum likelihood training.
• Score: 7.149677544861951
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Conditional Neural Processes (CNP; Garnelo et al., 2018) are an attractive family of meta-learning models which produce well-calibrated predictions, enable fast inference at test time, and are trainable via a simple maximum likelihood procedure. A limitation of CNPs is their inability to model dependencies in the outputs. This significantly hurts predictive performance and renders it impossible to draw coherent function samples, which limits the applicability of CNPs in down-stream applications and decision making. Neural Processes (NPs; Garnelo et al., 2018) attempt to alleviate this issue by using latent variables, relying on these to model output dependencies, but introduces difficulties stemming from approximate inference. One recent alternative (Bruinsma et al.,2021), which we refer to as the FullConvGNP, models dependencies in the predictions while still being trainable via exact maximum-likelihood. Unfortunately, the FullConvGNP relies on expensive 2D-dimensional convolutions, which limit its applicability to only one-dimensional data. In this work, we present an alternative way to model output dependencies which also lends itself maximum likelihood training but, unlike the FullConvGNP, can be scaled to two- and three-dimensional data. The proposed models exhibit good performance in synthetic experiments.

Related papers

• Computation-Aware Gaussian Processes: Model Selection And Linear-Time Inference [55.150117654242706]
  We show that model selection for computation-aware GPs trained on 1.8 million data points can be done within a few hours on a single GPU. As a result of this work, Gaussian processes can be trained on large-scale datasets without significantly compromising their ability to quantify uncertainty.
  arXiv  Detail & Related papers  (2024-11-01T21:11:48Z)
• Beyond Closure Models: Learning Chaotic-Systems via Physics-Informed Neural Operators [78.64101336150419]
  Predicting the long-term behavior of chaotic systems is crucial for various applications such as climate modeling. An alternative approach to such a full-resolved simulation is using a coarse grid and then correcting its errors through a temporalittext model. We propose an alternative end-to-end learning approach using a physics-informed neural operator (PINO) that overcomes this limitation.
  arXiv  Detail & Related papers  (2024-08-09T17:05:45Z)
• A Pseudo-Semantic Loss for Autoregressive Models with Logical Constraints [87.08677547257733]
  Neuro-symbolic AI bridges the gap between purely symbolic and neural approaches to learning. We show how to maximize the likelihood of a symbolic constraint w.r.t the neural network's output distribution. We also evaluate our approach on Sudoku and shortest-path prediction cast as autoregressive generation.
  arXiv  Detail & Related papers  (2023-12-06T20:58:07Z)
• Practical Equivariances via Relational Conditional Neural Processes [20.192899181958264]
  Conditional Neural Processes (CNPs) are a class of Conditional meta models popular for combining efficiency of amortized quantification with reliable uncertainty. We propose Conditional Neural Processes (RCNPs) as an effective approach to incorporate equivariances into any neural process model. Our proposed method extends the applicability and impact of equivariant processes to higher dimensions.
  arXiv  Detail & Related papers  (2023-06-19T13:27:27Z)
• Autoregressive Conditional Neural Processes [20.587835119831595]
  Conditional neural processes (CNPs) are attractive meta-learning models. They produce well-calibrated predictions and are trainable via a simple maximum likelihood procedure. CNPs are unable to model dependencies in their predictions. We propose to change how CNPs are deployed at test time, without any modifications to the model or training procedure.
  arXiv  Detail & Related papers  (2023-03-25T13:34:12Z)
• Transformer Neural Processes: Uncertainty-Aware Meta Learning Via Sequence Modeling [26.377099481072992]
  We propose Transformer Neural Processes (TNPs) for uncertainty-aware meta learning. We learn TNPs via an autoregressive likelihood-based objective and instantiate it with a novel transformer-based architecture. We show that TNPs achieve state-of-the-art performance on various benchmark problems.
  arXiv  Detail & Related papers  (2022-07-09T02:28:58Z)
• Practical Conditional Neural Processes Via Tractable Dependent Predictions [25.15531845287349]
  Conditional Neural Processes (CNPs) are meta-learning models which leverage the flexibility of deep learning to produce well-calibrated predictions. CNPs do not produce correlated predictions, making them inappropriate for many estimation and decision making tasks. We present a new class of Neural Process models that make correlated predictions and support exact maximum likelihood training.
  arXiv  Detail & Related papers  (2022-03-16T17:37:41Z)
• Closed-form Continuous-Depth Models [99.40335716948101]
  Continuous-depth neural models rely on advanced numerical differential equation solvers. We present a new family of models, termed Closed-form Continuous-depth (CfC) networks, that are simple to describe and at least one order of magnitude faster.
  arXiv  Detail & Related papers  (2021-06-25T22:08:51Z)
• TSNAT: Two-Step Non-Autoregressvie Transformer Models for Speech Recognition [69.68154370877615]
  The non-autoregressive (NAR) models can get rid of the temporal dependency between the output tokens and predict the entire output tokens in at least one step. To address these two problems, we propose a new model named the two-step non-autoregressive transformer(TSNAT) The results show that the TSNAT can achieve a competitive performance with the AR model and outperform many complicated NAR models.
  arXiv  Detail & Related papers  (2021-04-04T02:34:55Z)
• Bootstrapping Neural Processes [114.97111530885093]
  Neural Processes (NPs) implicitly define a broad class of processes with neural networks. NPs still rely on an assumption that uncertainty in processes is modeled by a single latent variable. We propose the Boostrapping Neural Process (BNP), a novel extension of the NP family using the bootstrap.
  arXiv  Detail & Related papers  (2020-08-07T02:23:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.