Meta-Learning Conjugate Priors for Few-Shot Bayesian Optimization

• URL: http://arxiv.org/abs/2101.00729v1
• Date: Sun, 3 Jan 2021 23:58:32 GMT
• Title: Meta-Learning Conjugate Priors for Few-Shot Bayesian Optimization
• Authors: Ruduan Plug
• Abstract summary: We propose a novel approach to utilize meta-learning to automate the estimation of informative conjugate prior distributions. From this process we generate priors that require only few data to estimate the shape parameters of the original distribution of the data.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Bayesian Optimization is methodology used in statistical modelling that utilizes a Gaussian process prior distribution to iteratively update a posterior distribution towards the true distribution of the data. Finding unbiased informative priors to sample from is challenging and can greatly influence the outcome on the posterior distribution if only few data are available. In this paper we propose a novel approach to utilize meta-learning to automate the estimation of informative conjugate prior distributions given a distribution class. From this process we generate priors that require only few data to estimate the shape parameters of the original distribution of the data.

Related papers

• Tackling the Problem of Distributional Shifts: Correcting Misspecified, High-Dimensional Data-Driven Priors for Inverse Problems [39.58317527488534]
  Data-driven population-level distributions are emerging as an appealing alternative to simple parametric priors in inverse problems. It is difficult to acquire independent and identically distributed samples from the underlying data-generating process of interest to train these models. We show that starting from a misspecified prior distribution, the updated distribution becomes progressively closer to the underlying population-level distribution.
  arXiv  Detail & Related papers  (2024-07-24T22:39:27Z)
• Sourcerer: Sample-based Maximum Entropy Source Distribution Estimation [5.673617376471343]
  We propose an approach which targets the maximum entropy distribution, i.e., prioritizes retaining as much uncertainty as possible. Our method is purely sample-based - leveraging the Sliced-Wasserstein distance to measure the discrepancy between the dataset and simulations. To demonstrate the utility of our approach, we infer source distributions for parameters of the Hodgkin-Huxley model from experimental datasets with thousands of single-neuron measurements.
  arXiv  Detail & Related papers  (2024-02-12T17:13:02Z)
• Distributed Markov Chain Monte Carlo Sampling based on the Alternating Direction Method of Multipliers [143.6249073384419]
  In this paper, we propose a distributed sampling scheme based on the alternating direction method of multipliers. We provide both theoretical guarantees of our algorithm's convergence and experimental evidence of its superiority to the state-of-the-art. In simulation, we deploy our algorithm on linear and logistic regression tasks and illustrate its fast convergence compared to existing gradient-based methods.
  arXiv  Detail & Related papers  (2024-01-29T02:08:40Z)
• Distribution Shift Inversion for Out-of-Distribution Prediction [57.22301285120695]
  We propose a portable Distribution Shift Inversion algorithm for Out-of-Distribution (OoD) prediction. We show that our method provides a general performance gain when plugged into a wide range of commonly used OoD algorithms.
  arXiv  Detail & Related papers  (2023-06-14T08:00:49Z)
• Open-Sampling: Exploring Out-of-Distribution data for Re-balancing Long-tailed datasets [24.551465814633325]
  Deep neural networks usually perform poorly when the training dataset suffers from extreme class imbalance. Recent studies found that directly training with out-of-distribution data in a semi-supervised manner would harm the generalization performance. We propose a novel method called Open-sampling, which utilizes open-set noisy labels to re-balance the class priors of the training dataset.
  arXiv  Detail & Related papers  (2022-06-17T14:29:52Z)
• Invariance Learning in Deep Neural Networks with Differentiable Laplace Approximations [76.82124752950148]
  We develop a convenient gradient-based method for selecting the data augmentation. We use a differentiable Kronecker-factored Laplace approximation to the marginal likelihood as our objective.
  arXiv  Detail & Related papers  (2022-02-22T02:51:11Z)
• Scalable Marginal Likelihood Estimation for Model Selection in Deep Learning [78.83598532168256]
  Marginal-likelihood based model-selection is rarely used in deep learning due to estimation difficulties. Our work shows that marginal likelihoods can improve generalization and be useful when validation data is unavailable.
  arXiv  Detail & Related papers  (2021-04-11T09:50:24Z)
• Source-free Domain Adaptation via Distributional Alignment by Matching Batch Normalization Statistics [85.75352990739154]
  We propose a novel domain adaptation method for the source-free setting. We use batch normalization statistics stored in the pretrained model to approximate the distribution of unobserved source data. Our method achieves competitive performance with state-of-the-art domain adaptation methods.
  arXiv  Detail & Related papers  (2021-01-19T14:22:33Z)
• The Bayesian Method of Tensor Networks [1.7894377200944511]
  We study the Bayesian framework of the Network from two perspective. We study the Bayesian properties of the Network by visualizing the parameters of the model and the decision boundaries in the two dimensional synthetic data set.
  arXiv  Detail & Related papers  (2021-01-01T14:59:15Z)
• Model Fusion with Kullback--Leibler Divergence [58.20269014662046]
  We propose a method to fuse posterior distributions learned from heterogeneous datasets. Our algorithm relies on a mean field assumption for both the fused model and the individual dataset posteriors.
  arXiv  Detail & Related papers  (2020-07-13T03:27:45Z)
• Domain Adaptive Bootstrap Aggregating [5.444459446244819]
  bootstrap aggregating, or bagging, is a popular method for improving stability of predictive algorithms. This article proposes a domain adaptive bagging method coupled with a new iterative nearest neighbor sampler.
  arXiv  Detail & Related papers  (2020-01-12T20:02:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.