Martingale Posterior Neural Processes
- URL: http://arxiv.org/abs/2304.09431v1
- Date: Wed, 19 Apr 2023 05:58:18 GMT
- Title: Martingale Posterior Neural Processes
- Authors: Hyungi Lee, Eunggu Yun, Giung Nam, Edwin Fong, Juho Lee
- Abstract summary: A Neural Process (NP) estimates a process implicitly defined with neural networks given a stream of data.
We take a different approach based on the martingale posterior, a recently developed alternative to Bayesian inference.
We show that the uncertainty in the generated future data actually corresponds to the uncertainty of the implicitly defined Bayesian posteriors.
- Score: 14.913697718688931
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: A Neural Process (NP) estimates a stochastic process implicitly defined with
neural networks given a stream of data, rather than pre-specifying priors
already known, such as Gaussian processes. An ideal NP would learn everything
from data without any inductive biases, but in practice, we often restrict the
class of stochastic processes for the ease of estimation. One such restriction
is the use of a finite-dimensional latent variable accounting for the
uncertainty in the functions drawn from NPs. Some recent works show that this
can be improved with more "data-driven" source of uncertainty such as
bootstrapping. In this work, we take a different approach based on the
martingale posterior, a recently developed alternative to Bayesian inference.
For the martingale posterior, instead of specifying prior-likelihood pairs, a
predictive distribution for future data is specified. Under specific conditions
on the predictive distribution, it can be shown that the uncertainty in the
generated future data actually corresponds to the uncertainty of the implicitly
defined Bayesian posteriors. Based on this result, instead of assuming any form
of the latent variables, we equip a NP with a predictive distribution
implicitly defined with neural networks and use the corresponding martingale
posteriors as the source of uncertainty. The resulting model, which we name as
Martingale Posterior Neural Process (MPNP), is demonstrated to outperform
baselines on various tasks.
Related papers
- Rényi Neural Processes [14.11793373584558]
We propose R'enyi Neural Processes (RNP) to ameliorate the impacts of prior misspecification.
We scale the density ratio $fracpq$ by the power of (1-$alpha$) in the divergence gradients with respect to the posterior.
Our experiments show consistent log-likelihood improvements over state-of-the-art NP family models.
arXiv Detail & Related papers (2024-05-25T00:14:55Z) - Posterior Uncertainty Quantification in Neural Networks using Data Augmentation [3.9860047080844807]
We show that deep ensembling is a fundamentally mis-specified model class, since it assumes that future data are supported on existing observations only.
We propose MixupMP, a method that constructs a more realistic predictive distribution using popular data augmentation techniques.
Our empirical analysis showcases that MixupMP achieves superior predictive performance and uncertainty quantification on various image classification datasets.
arXiv Detail & Related papers (2024-03-18T17:46:07Z) - Tractable Function-Space Variational Inference in Bayesian Neural
Networks [72.97620734290139]
A popular approach for estimating the predictive uncertainty of neural networks is to define a prior distribution over the network parameters.
We propose a scalable function-space variational inference method that allows incorporating prior information.
We show that the proposed method leads to state-of-the-art uncertainty estimation and predictive performance on a range of prediction tasks.
arXiv Detail & Related papers (2023-12-28T18:33:26Z) - Looking at the posterior: accuracy and uncertainty of neural-network
predictions [0.0]
We show that prediction accuracy depends on both epistemic and aleatoric uncertainty.
We introduce a novel acquisition function that outperforms common uncertainty-based methods.
arXiv Detail & Related papers (2022-11-26T16:13:32Z) - Federated Learning with Uncertainty via Distilled Predictive
Distributions [14.828509220023387]
We present a framework for federated learning with uncertainty where, in each round, each client infers the posterior distribution over its parameters as well as the posterior predictive distribution (PPD)
Unlike some of the recent Bayesian approaches to federated learning, our approach does not require sending the whole posterior distribution of the parameters from each client to the server.
Our approach does not make any restrictive assumptions, such as the form of the clients' posterior distributions, or of their PPDs.
arXiv Detail & Related papers (2022-06-15T14:24:59Z) - NUQ: Nonparametric Uncertainty Quantification for Deterministic Neural
Networks [151.03112356092575]
We show the principled way to measure the uncertainty of predictions for a classifier based on Nadaraya-Watson's nonparametric estimate of the conditional label distribution.
We demonstrate the strong performance of the method in uncertainty estimation tasks on a variety of real-world image datasets.
arXiv Detail & Related papers (2022-02-07T12:30:45Z) - Dense Uncertainty Estimation [62.23555922631451]
In this paper, we investigate neural networks and uncertainty estimation techniques to achieve both accurate deterministic prediction and reliable uncertainty estimation.
We work on two types of uncertainty estimations solutions, namely ensemble based methods and generative model based methods, and explain their pros and cons while using them in fully/semi/weakly-supervised framework.
arXiv Detail & Related papers (2021-10-13T01:23:48Z) - When in Doubt: Neural Non-Parametric Uncertainty Quantification for
Epidemic Forecasting [70.54920804222031]
Most existing forecasting models disregard uncertainty quantification, resulting in mis-calibrated predictions.
Recent works in deep neural models for uncertainty-aware time-series forecasting also have several limitations.
We model the forecasting task as a probabilistic generative process and propose a functional neural process model called EPIFNP.
arXiv Detail & Related papers (2021-06-07T18:31:47Z) - Bootstrapping Neural Processes [114.97111530885093]
Neural Processes (NPs) implicitly define a broad class of processes with neural networks.
NPs still rely on an assumption that uncertainty in processes is modeled by a single latent variable.
We propose the Boostrapping Neural Process (BNP), a novel extension of the NP family using the bootstrap.
arXiv Detail & Related papers (2020-08-07T02:23:34Z) - Unlabelled Data Improves Bayesian Uncertainty Calibration under
Covariate Shift [100.52588638477862]
We develop an approximate Bayesian inference scheme based on posterior regularisation.
We demonstrate the utility of our method in the context of transferring prognostic models of prostate cancer across globally diverse populations.
arXiv Detail & Related papers (2020-06-26T13:50:19Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.