Mixture Density Network Estimation of Continuous Variable Maximum Likelihood Using Discrete Training Samples

• URL: http://arxiv.org/abs/2103.13416v1
• Date: Wed, 24 Mar 2021 18:02:55 GMT
• Title: Mixture Density Network Estimation of Continuous Variable Maximum Likelihood Using Discrete Training Samples
• Authors: Charles Burton, Spencer Stubbs, Peter Onyisi
• Abstract summary: Mixture Density Networks (MDNs) can be used to generate probability density functions of model parameters $boldsymboltheta$ given a set of observables. We demonstrate the usage of MDNs for parameter estimation, discuss the origins of the biases, and propose a corrective method for each issue.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Mixture Density Networks (MDNs) can be used to generate probability density functions of model parameters $\boldsymbol{\theta}$ given a set of observables $\mathbf{x}$. In some applications, training data are available only for discrete values of a continuous parameter $\boldsymbol{\theta}$. In such situations a number of performance-limiting issues arise which can result in biased estimates. We demonstrate the usage of MDNs for parameter estimation, discuss the origins of the biases, and propose a corrective method for each issue.

Related papers

• Denoising Score Matching with Random Features: Insights on Diffusion Models from Precise Learning Curves [8.539326630369592]
  We derive precise expressions for test and train errors of denoising score matching in generative diffusion models. We operate in a regime where the dimension $d$, number of data samples $n$, and number of features $p$ tend to infinity. Our work sheds light on the conditions enhancing either generalization or memorization.
  arXiv  Detail & Related papers  (2025-02-01T06:43:33Z)
• Distributed Nonparametric Estimation: from Sparse to Dense Samples per Terminal [9.766173684831324]
  We characterize the minimax optimal rates for all regimes, and identify phase transitions of the optimal rates as the samples per terminal vary from sparse to dense. This fully solves the problem left open by previous works, whose scopes are limited to regimes with either dense samples or a single sample per terminal. The optimal rates are immediate for various special cases such as density estimation, Gaussian, binary, Poisson and heteroskedastic regression models.
  arXiv  Detail & Related papers  (2025-01-14T06:41:55Z)
• A quasi-Bayesian sequential approach to deconvolution density estimation [7.10052009802944]
  Density deconvolution addresses the estimation of the unknown density function $f$ of a random signal from data. We consider the problem of density deconvolution in a streaming or online setting where noisy data arrive progressively. By relying on a quasi-Bayesian sequential approach, we obtain estimates of $f$ that are of easy evaluation.
  arXiv  Detail & Related papers  (2024-08-26T16:40:04Z)
• $χ$SPN: Characteristic Interventional Sum-Product Networks for Causal Inference in Hybrid Domains [19.439265962277716]
  We propose aCharacteristic Interventional Sum-Product Network ($chi$SPN) that is capable of estimating interventional distributions in presence of random variables. $chi$SPN uses characteristic functions in the leaves of an interventional SPN (iSPN) thereby providing a unified view for discrete and continuous random variables. A neural network is used to estimate the parameters of the learned iSPN using the intervened data.
  arXiv  Detail & Related papers  (2024-08-14T13:31:32Z)
• Probabilistic Classification by Density Estimation Using Gaussian Mixture Model and Masked Autoregressive Flow [1.3706331473063882]
  Density estimation, which estimates the distribution of data, is an important category of probabilistic machine learning. In this paper, we use the density estimators for classification, although they are often used for estimating the distribution of data. We model the likelihood of classes of data by density estimation, specifically using GMM and MAF.
  arXiv  Detail & Related papers  (2023-10-16T21:37:22Z)
• On the Dynamics of Inference and Learning [0.0]
  We present a treatment of this Bayesian updating process as a continuous dynamical system. We show that when the Cram'er-Rao bound is saturated the learning rate is governed by a simple $1/T$ power-law.
  arXiv  Detail & Related papers  (2022-04-19T18:04:36Z)
• NUQ: Nonparametric Uncertainty Quantification for Deterministic Neural Networks [151.03112356092575]
  We show the principled way to measure the uncertainty of predictions for a classifier based on Nadaraya-Watson's nonparametric estimate of the conditional label distribution. We demonstrate the strong performance of the method in uncertainty estimation tasks on a variety of real-world image datasets.
  arXiv  Detail & Related papers  (2022-02-07T12:30:45Z)
• Inverting brain grey matter models with likelihood-free inference: a tool for trustable cytoarchitecture measurements [62.997667081978825]
  characterisation of the brain grey matter cytoarchitecture with quantitative sensitivity to soma density and volume remains an unsolved challenge in dMRI. We propose a new forward model, specifically a new system of equations, requiring a few relatively sparse b-shells. We then apply modern tools from Bayesian analysis known as likelihood-free inference (LFI) to invert our proposed model.
  arXiv  Detail & Related papers  (2021-11-15T09:08:27Z)
• Sampling-free Variational Inference for Neural Networks with Multiplicative Activation Noise [51.080620762639434]
  We propose a more efficient parameterization of the posterior approximation for sampling-free variational inference. Our approach yields competitive results for standard regression problems and scales well to large-scale image classification tasks.
  arXiv  Detail & Related papers  (2021-03-15T16:16:18Z)
• A Multi-Channel Neural Graphical Event Model with Negative Evidence [76.51278722190607]
  Event datasets are sequences of events of various types occurring irregularly over the time-line. We propose a non-parametric deep neural network approach in order to estimate the underlying intensity functions.
  arXiv  Detail & Related papers  (2020-02-21T23:10:50Z)
• Stochastic Normalizing Flows [52.92110730286403]
  We introduce normalizing flows for maximum likelihood estimation and variational inference (VI) using differential equations (SDEs) Using the theory of rough paths, the underlying Brownian motion is treated as a latent variable and approximated, enabling efficient training of neural SDEs. These SDEs can be used for constructing efficient chains to sample from the underlying distribution of a given dataset.
  arXiv  Detail & Related papers  (2020-02-21T20:47:55Z)
• Posterior Ratio Estimation of Latent Variables [14.619879849533662]
  In some applications, we want to compare distributions of random variables that are emphinferred from observations. We study the problem of estimating the ratio between two posterior probability density functions of a latent variable.
  arXiv  Detail & Related papers  (2020-02-15T16:46:42Z)
• Algebraic and Analytic Approaches for Parameter Learning in Mixture Models [66.96778152993858]
  We present two different approaches for parameter learning in several mixture models in one dimension. For some of these distributions, our results represent the first guarantees for parameter estimation.
  arXiv  Detail & Related papers  (2020-01-19T05:10:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.