Fast Maximum Likelihood Estimation and Supervised Classification for the Beta-Liouville Multinomial

• URL: http://arxiv.org/abs/2006.07454v1
• Date: Fri, 12 Jun 2020 20:30:12 GMT
• Title: Fast Maximum Likelihood Estimation and Supervised Classification for the Beta-Liouville Multinomial
• Authors: Steven Michael Lakin, Zaid Abdo
• Abstract summary: We show that the Beta-Liouville multinomial is comparable in efficiency to the Dirichlet multinomial for Newton-Raphson maximum likelihood estimation. We also demonstrate that the Beta-Liouville multinomial outperforms the multinomial and Dirichlet multinomial on two out of four gold standard datasets.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The multinomial and related distributions have long been used to model categorical, count-based data in fields ranging from bioinformatics to natural language processing. Commonly utilized variants include the standard multinomial and the Dirichlet multinomial distributions due to their computational efficiency and straightforward parameter estimation process. However, these distributions make strict assumptions about the mean, variance, and covariance between the categorical features being modeled. If these assumptions are not met by the data, it may result in poor parameter estimates and loss in accuracy for downstream applications like classification. Here, we explore efficient parameter estimation and supervised classification methods using an alternative distribution, called the Beta-Liouville multinomial, which relaxes some of the multinomial assumptions. We show that the Beta-Liouville multinomial is comparable in efficiency to the Dirichlet multinomial for Newton-Raphson maximum likelihood estimation, and that its performance on simulated data matches or exceeds that of the multinomial and Dirichlet multinomial distributions. Finally, we demonstrate that the Beta-Liouville multinomial outperforms the multinomial and Dirichlet multinomial on two out of four gold standard datasets, supporting its use in modeling data with low to medium class overlap in a supervised classification context.

Related papers

• Hierarchical mixtures of Unigram models for short text clustering: the role of Beta-Liouville priors [1.03590082373586]
  This paper presents a variant of the Multinomial mixture model tailored for the unsupervised classification of short text data. We explore an alternative priorational--the Beta-Liouville distribution--which offers a more flexible correlation structure than the Dirichlet.
  arXiv  Detail & Related papers  (2024-10-29T08:56:29Z)
• Non-asymptotic approximations for Pearson's chi-square statistic and its application to confidence intervals for strictly convex functions of the probability weights of discrete distributions [0.0]
  We develop a non-asymptotic local normal approximation for multinomial probabilities. We apply our results to find confidence intervals for the negative entropy of discrete distributions.
  arXiv  Detail & Related papers  (2023-09-05T01:18:48Z)
• AdaCat: Adaptive Categorical Discretization for Autoregressive Models [84.85102013917606]
  We propose an efficient, expressive, multimodal parameterization called Adaptive Categorical Discretization (AdaCat) AdaCat discretizes each dimension of an autoregressive model adaptively, which allows the model to allocate density to fine intervals of interest.
  arXiv  Detail & Related papers  (2022-08-03T17:53:46Z)
• Uncertainty Modeling for Out-of-Distribution Generalization [56.957731893992495]
  We argue that the feature statistics can be properly manipulated to improve the generalization ability of deep learning models. Common methods often consider the feature statistics as deterministic values measured from the learned features. We improve the network generalization ability by modeling the uncertainty of domain shifts with synthesized feature statistics during training.
  arXiv  Detail & Related papers  (2022-02-08T16:09:12Z)
• Sampling from Arbitrary Functions via PSD Models [55.41644538483948]
  We take a two-step approach by first modeling the probability distribution and then sampling from that model. We show that these models can approximate a large class of densities concisely using few evaluations, and present a simple algorithm to effectively sample from these models.
  arXiv  Detail & Related papers  (2021-10-20T12:25:22Z)
• Scalable Marginal Likelihood Estimation for Model Selection in Deep Learning [78.83598532168256]
  Marginal-likelihood based model-selection is rarely used in deep learning due to estimation difficulties. Our work shows that marginal likelihoods can improve generalization and be useful when validation data is unavailable.
  arXiv  Detail & Related papers  (2021-04-11T09:50:24Z)
• Asymptotically Exact and Fast Gaussian Copula Models for Imputation of Mixed Data Types [0.13764085113103217]
  Missing values with mixed data types is a common problem in a large number of machine learning applications. Gustafson copula models have been suggested as a means of performing imputation of missing values using a probabilistic framework. We address the first limitation using direct and arbitrarily precise approximations both for model estimation and imputation by using randomized quasi-Monte Carlo procedures.
  arXiv  Detail & Related papers  (2021-02-04T14:42:29Z)
• A Class of Conjugate Priors for Multinomial Probit Models which Includes the Multivariate Normal One [0.3553493344868413]
  We show that the entire class of unified skew-normal (SUN) distributions is conjugate to several multinomial probit models. We improve upon state-of-the-art solutions for posterior inference and classification.
  arXiv  Detail & Related papers  (2020-07-14T10:08:23Z)
• SUMO: Unbiased Estimation of Log Marginal Probability for Latent Variable Models [80.22609163316459]
  We introduce an unbiased estimator of the log marginal likelihood and its gradients for latent variable models based on randomized truncation of infinite series. We show that models trained using our estimator give better test-set likelihoods than a standard importance-sampling based approach for the same average computational cost.
  arXiv  Detail & Related papers  (2020-04-01T11:49:30Z)
• Review of Probability Distributions for Modeling Count Data [0.0]
  Generalized linear models enable direct modeling of counts in a regression context. When counts contain only relative information, multinomial or Dirichlet-multinomial models can be more appropriate.
  arXiv  Detail & Related papers  (2020-01-10T18:28:19Z)
• CatBoostLSS -- An extension of CatBoost to probabilistic forecasting [91.3755431537592]
  We propose a new framework that predicts the entire conditional distribution of a univariable response variable. CatBoostLSS models all moments of a parametric distribution instead of the conditional mean only. We present both a simulation study and real-world examples that demonstrate the benefits of our approach.
  arXiv  Detail & Related papers  (2020-01-04T15:42:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.