A similarity-based Bayesian mixture-of-experts model
- URL: http://arxiv.org/abs/2012.02130v2
- Date: Sun, 9 May 2021 10:14:23 GMT
- Title: A similarity-based Bayesian mixture-of-experts model
- Authors: Tianfang Zhang and Rasmus Bokrantz and Jimmy Olsson
- Abstract summary: We present a new non-parametric mixture-of-experts model for multivariate regression problems.
Using a conditionally specified model, predictions for out-of-sample inputs are based on similarities to each observed data point.
Posterior inference is performed on the parameters of the mixture as well as the distance metric.
- Score: 0.5156484100374058
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We present a new nonparametric mixture-of-experts model for multivariate
regression problems, inspired by the probabilistic $k$-nearest neighbors
algorithm. Using a conditionally specified model, predictions for out-of-sample
inputs are based on similarities to each observed data point, yielding
predictive distributions represented by Gaussian mixtures. Posterior inference
is performed on the parameters of the mixture components as well as the
distance metric using a mean-field variational Bayes algorithm accompanied with
a stochastic gradient-based optimization procedure. The proposed method is
especially advantageous in settings where inputs are of relatively high
dimension in comparison to the data size, where input--output relationships are
complex, and where predictive distributions may be skewed or multimodal.
Computational studies on two synthetic datasets and one dataset comprising dose
statistics of radiation therapy treatment plans show that our
mixture-of-experts method performs similarly or better than a conditional
Dirichlet process mixture model both in terms of validation metrics and visual
inspection.
Related papers
- Mixup Regularization: A Probabilistic Perspective [11.501663622998697]
We introduce a novel framework for mixup regularization based on probabilistic fusion.
For data distributed according to a member of the exponential family, we show that likelihood functions can be analytically fused.
We propose an extension of probabilistic mixup, which allows for fusion of inputs at an arbitrary intermediate layer of the neural network.
arXiv Detail & Related papers (2025-02-19T15:39:14Z) - Collaborative Heterogeneous Causal Inference Beyond Meta-analysis [68.4474531911361]
We propose a collaborative inverse propensity score estimator for causal inference with heterogeneous data.
Our method shows significant improvements over the methods based on meta-analysis when heterogeneity increases.
arXiv Detail & Related papers (2024-04-24T09:04:36Z) - Diffusion posterior sampling for simulation-based inference in tall data settings [53.17563688225137]
Simulation-based inference ( SBI) is capable of approximating the posterior distribution that relates input parameters to a given observation.
In this work, we consider a tall data extension in which multiple observations are available to better infer the parameters of the model.
We compare our method to recently proposed competing approaches on various numerical experiments and demonstrate its superiority in terms of numerical stability and computational cost.
arXiv Detail & Related papers (2024-04-11T09:23:36Z) - Partially factorized variational inference for high-dimensional mixed models [0.0]
Variational inference is a popular way to perform such computations, especially in the Bayesian context.
We show that standard mean-field variational inference dramatically underestimates posterior uncertainty in high-dimensions.
We then show how appropriately relaxing the mean-field assumption leads to methods whose uncertainty quantification does not deteriorate in high-dimensions.
arXiv Detail & Related papers (2023-12-20T16:12:37Z) - Estimating Joint Probability Distribution With Low-Rank Tensor
Decomposition, Radon Transforms and Dictionaries [3.0892724364965005]
We describe a method for estimating the joint probability density from data samples by assuming that the underlying distribution can be decomposed as a mixture of product densities with few mixture components.
We combine two key ideas: dictionaries to represent 1-D densities, and random projections to estimate the joint distribution from 1-D marginals.
Our algorithm benefits from improved sample complexity over the previous dictionary-based approach by using 1-D marginals for reconstruction.
arXiv Detail & Related papers (2023-04-18T05:37:15Z) - Bayesian Sparse Regression for Mixed Multi-Responses with Application to
Runtime Metrics Prediction in Fog Manufacturing [6.288767115532775]
Fog manufacturing can greatly enhance traditional manufacturing systems through distributed computation Fog units.
It is known that the predictive offloading methods highly depend on accurate prediction and uncertainty quantification of runtime performance metrics.
We propose a Bayesian sparse regression for multivariate mixed responses to enhance the prediction of runtime performance metrics.
arXiv Detail & Related papers (2022-10-10T16:14:08Z) - A Robust and Flexible EM Algorithm for Mixtures of Elliptical
Distributions with Missing Data [71.9573352891936]
This paper tackles the problem of missing data imputation for noisy and non-Gaussian data.
A new EM algorithm is investigated for mixtures of elliptical distributions with the property of handling potential missing data.
Experimental results on synthetic data demonstrate that the proposed algorithm is robust to outliers and can be used with non-Gaussian data.
arXiv Detail & Related papers (2022-01-28T10:01:37Z) - MINIMALIST: Mutual INformatIon Maximization for Amortized Likelihood
Inference from Sampled Trajectories [61.3299263929289]
Simulation-based inference enables learning the parameters of a model even when its likelihood cannot be computed in practice.
One class of methods uses data simulated with different parameters to infer an amortized estimator for the likelihood-to-evidence ratio.
We show that this approach can be formulated in terms of mutual information between model parameters and simulated data.
arXiv Detail & Related papers (2021-06-03T12:59:16Z) - Fitting Laplacian Regularized Stratified Gaussian Models [0.0]
We consider the problem of jointly estimating multiple related zero-mean Gaussian distributions from data.
We propose a distributed method that scales to large problems, and illustrate the efficacy of the method with examples in finance, radar signal processing, and weather forecasting.
arXiv Detail & Related papers (2020-05-04T18:00:59Z) - Machine learning for causal inference: on the use of cross-fit
estimators [77.34726150561087]
Doubly-robust cross-fit estimators have been proposed to yield better statistical properties.
We conducted a simulation study to assess the performance of several estimators for the average causal effect (ACE)
When used with machine learning, the doubly-robust cross-fit estimators substantially outperformed all of the other estimators in terms of bias, variance, and confidence interval coverage.
arXiv Detail & Related papers (2020-04-21T23:09:55Z) - Asymptotic Analysis of an Ensemble of Randomly Projected Linear
Discriminants [94.46276668068327]
In [1], an ensemble of randomly projected linear discriminants is used to classify datasets.
We develop a consistent estimator of the misclassification probability as an alternative to the computationally-costly cross-validation estimator.
We also demonstrate the use of our estimator for tuning the projection dimension on both real and synthetic data.
arXiv Detail & Related papers (2020-04-17T12:47:04Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.