Evidential Conditional Neural Processes
- URL: http://arxiv.org/abs/2212.00131v1
- Date: Wed, 30 Nov 2022 21:50:55 GMT
- Title: Evidential Conditional Neural Processes
- Authors: Deep Shankar Pandey and Qi Yu
- Abstract summary: Conditional Neural Process (CNP) models offer a promising direction to tackle few-shot problems.
Current CNP models only capture the overall uncertainty for the prediction made on a target data point.
We propose Evidential Conditional Neural Processes (ECNP), which replace the standard Gaussian distribution used by CNP.
- Score: 7.257751371276488
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The Conditional Neural Process (CNP) family of models offer a promising
direction to tackle few-shot problems by achieving better scalability and
competitive predictive performance. However, the current CNP models only
capture the overall uncertainty for the prediction made on a target data point.
They lack a systematic fine-grained quantification on the distinct sources of
uncertainty that are essential for model training and decision-making under the
few-shot setting. We propose Evidential Conditional Neural Processes (ECNP),
which replace the standard Gaussian distribution used by CNP with a much richer
hierarchical Bayesian structure through evidential learning to achieve
epistemic-aleatoric uncertainty decomposition. The evidential hierarchical
structure also leads to a theoretically justified robustness over noisy
training tasks. Theoretical analysis on the proposed ECNP establishes the
relationship with CNP while offering deeper insights on the roles of the
evidential parameters. Extensive experiments conducted on both synthetic and
real-world data demonstrate the effectiveness of our proposed model in various
few-shot settings.
Related papers
- Parameter-Efficient Fine-Tuning for Continual Learning: A Neural Tangent Kernel Perspective [125.00228936051657]
We introduce NTK-CL, a novel framework that eliminates task-specific parameter storage while adaptively generating task-relevant features.
By fine-tuning optimizable parameters with appropriate regularization, NTK-CL achieves state-of-the-art performance on established PEFT-CL benchmarks.
arXiv Detail & Related papers (2024-07-24T09:30:04Z) - A PAC-Bayesian Perspective on the Interpolating Information Criterion [54.548058449535155]
We show how a PAC-Bayes bound is obtained for a general class of models, characterizing factors which influence performance in the interpolating regime.
We quantify how the test error for overparameterized models achieving effectively zero training error depends on the quality of the implicit regularization imposed by e.g. the combination of model, parameter-initialization scheme.
arXiv Detail & Related papers (2023-11-13T01:48:08Z) - Variational Inference for Bayesian Neural Networks under Model and
Parameter Uncertainty [12.211659310564425]
We apply the concept of model uncertainty as a framework for structural learning in BNNs.
We suggest an adaptation of a scalable variational inference approach with reparametrization of marginal inclusion probabilities.
arXiv Detail & Related papers (2023-05-01T16:38:17Z) - Autoregressive Conditional Neural Processes [20.587835119831595]
Conditional neural processes (CNPs) are attractive meta-learning models.
They produce well-calibrated predictions and are trainable via a simple maximum likelihood procedure.
CNPs are unable to model dependencies in their predictions.
We propose to change how CNPs are deployed at test time, without any modifications to the model or training procedure.
arXiv Detail & Related papers (2023-03-25T13:34:12Z) - Latent Variable Representation for Reinforcement Learning [131.03944557979725]
It remains unclear theoretically and empirically how latent variable models may facilitate learning, planning, and exploration to improve the sample efficiency of model-based reinforcement learning.
We provide a representation view of the latent variable models for state-action value functions, which allows both tractable variational learning algorithm and effective implementation of the optimism/pessimism principle.
In particular, we propose a computationally efficient planning algorithm with UCB exploration by incorporating kernel embeddings of latent variable models.
arXiv Detail & Related papers (2022-12-17T00:26:31Z) - The Unreasonable Effectiveness of Deep Evidential Regression [72.30888739450343]
A new approach with uncertainty-aware regression-based neural networks (NNs) shows promise over traditional deterministic methods and typical Bayesian NNs.
We detail the theoretical shortcomings and analyze the performance on synthetic and real-world data sets, showing that Deep Evidential Regression is a quantification rather than an exact uncertainty.
arXiv Detail & Related papers (2022-05-20T10:10:32Z) - When in Doubt: Neural Non-Parametric Uncertainty Quantification for
Epidemic Forecasting [70.54920804222031]
Most existing forecasting models disregard uncertainty quantification, resulting in mis-calibrated predictions.
Recent works in deep neural models for uncertainty-aware time-series forecasting also have several limitations.
We model the forecasting task as a probabilistic generative process and propose a functional neural process model called EPIFNP.
arXiv Detail & Related papers (2021-06-07T18:31:47Z) - The Gaussian Neural Process [39.81327564209865]
We provide a rigorous analysis of the standard maximum-likelihood objective used to train conditional NPs.
We propose a new member to the Neural Process family called the Neural Process (GNP), which models predictive correlations, incorporates translation, provides universal approximation guarantees, and demonstrates encouraging performance.
arXiv Detail & Related papers (2021-01-10T19:15:27Z) - Message Passing Neural Processes [3.0969191504482247]
We introduce Message Passing Neural Processes (MPNPs), which explicitly makes use of relational structure within the model.
MPNPs thrive at lower sampling rates, on existing benchmarks and newly-proposed CA and Cora-Branched tasks.
We report strong generalisation over density-based CA rulesets and significant gains in challenging arbitrary-labelling and few-shot learning setups.
arXiv Detail & Related papers (2020-09-29T09:40:09Z) - Multiplicative noise and heavy tails in stochastic optimization [62.993432503309485]
empirical optimization is central to modern machine learning, but its role in its success is still unclear.
We show that it commonly arises in parameters of discrete multiplicative noise due to variance.
A detailed analysis is conducted in which we describe on key factors, including recent step size, and data, all exhibit similar results on state-of-the-art neural network models.
arXiv Detail & Related papers (2020-06-11T09:58:01Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.