Bayesian Inference for Gamma Models

• URL: http://arxiv.org/abs/2106.01906v1
• Date: Thu, 3 Jun 2021 14:58:39 GMT
• Title: Bayesian Inference for Gamma Models
• Authors: Jingyu He, Nicholas Polson, Jianeng Xu
• Abstract summary: We use the theory of normal variance-mean mixtures to derive a data augmentation scheme for models that include gamma functions. We illustrate our methodology on a number of examples, including gamma shape inference, negative binomial regression and Dirichlet allocation.
• Score: 4.189643331553922
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We use the theory of normal variance-mean mixtures to derive a data augmentation scheme for models that include gamma functions. Our methodology applies to many situations in statistics and machine learning, including Multinomial-Dirichlet distributions, Negative binomial regression, Poisson-Gamma hierarchical models, Extreme value models, to name but a few. All of those models include a gamma function which does not admit a natural conjugate prior distribution providing a significant challenge to inference and prediction. To provide a data augmentation strategy, we construct and develop the theory of the class of Exponential Reciprocal Gamma distributions. This allows scalable EM and MCMC algorithms to be developed. We illustrate our methodology on a number of examples, including gamma shape inference, negative binomial regression and Dirichlet allocation. Finally, we conclude with directions for future research.

Related papers

• Influence Functions for Scalable Data Attribution in Diffusion Models [52.92223039302037]
  Diffusion models have led to significant advancements in generative modelling. Yet their widespread adoption poses challenges regarding data attribution and interpretability. In this paper, we aim to help address such challenges by developing an textitinfluence functions framework.
  arXiv  Detail & Related papers  (2024-10-17T17:59:02Z)
• Scaling and renormalization in high-dimensional regression [72.59731158970894]
  This paper presents a succinct derivation of the training and generalization performance of a variety of high-dimensional ridge regression models. We provide an introduction and review of recent results on these topics, aimed at readers with backgrounds in physics and deep learning.
  arXiv  Detail & Related papers  (2024-05-01T15:59:00Z)
• Unveil Conditional Diffusion Models with Classifier-free Guidance: A Sharp Statistical Theory [87.00653989457834]
  Conditional diffusion models serve as the foundation of modern image synthesis and find extensive application in fields like computational biology and reinforcement learning. Despite the empirical success, theory of conditional diffusion models is largely missing. This paper bridges the gap by presenting a sharp statistical theory of distribution estimation using conditional diffusion models.
  arXiv  Detail & Related papers  (2024-03-18T17:08:24Z)
• Deep Generative Modeling on Limited Data with Regularization by Nontransferable Pre-trained Models [32.52492468276371]
  We propose regularized deep generative model (Reg-DGM) to reduce the variance of generative modeling with limited data. Reg-DGM uses a pre-trained model to optimize a weighted sum of a certain divergence and the expectation of an energy function. Empirically, with various pre-trained feature extractors and a data-dependent energy function, Reg-DGM consistently improves the generation performance of strong DGMs with limited data.
  arXiv  Detail & Related papers  (2022-08-30T10:28:50Z)
• A Graphical Model for Fusing Diverse Microbiome Data [2.385985842958366]
  We introduce a flexible multinomial-Gaussian generative model for jointly modeling such count data. We present a computationally scalable variational Expectation-Maximization (EM) algorithm for inferring the latent variables and the parameters of the model.
  arXiv  Detail & Related papers  (2022-08-21T17:54:39Z)
• On the Generalization and Adaption Performance of Causal Models [99.64022680811281]
  Differentiable causal discovery has proposed to factorize the data generating process into a set of modules. We study the generalization and adaption performance of such modular neural causal models. Our analysis shows that the modular neural causal models outperform other models on both zero and few-shot adaptation in low data regimes.
  arXiv  Detail & Related papers  (2022-06-09T17:12:32Z)
• Riemannian Score-Based Generative Modeling [56.20669989459281]
  We introduce score-based generative models (SGMs) demonstrating remarkable empirical performance. Current SGMs make the underlying assumption that the data is supported on a Euclidean manifold with flat geometry. This prevents the use of these models for applications in robotics, geoscience or protein modeling.
  arXiv  Detail & Related papers  (2022-02-06T11:57:39Z)
• Latent Gaussian Model Boosting [0.0]
  Tree-boosting shows excellent predictive accuracy on many data sets. We obtain increased predictive accuracy compared to existing approaches in both simulated and real-world data experiments.
  arXiv  Detail & Related papers  (2021-05-19T07:36:30Z)
• A Class of Conjugate Priors for Multinomial Probit Models which Includes the Multivariate Normal One [0.3553493344868413]
  We show that the entire class of unified skew-normal (SUN) distributions is conjugate to several multinomial probit models. We improve upon state-of-the-art solutions for posterior inference and classification.
  arXiv  Detail & Related papers  (2020-07-14T10:08:23Z)
• Good Classifiers are Abundant in the Interpolating Regime [64.72044662855612]
  We develop a methodology to compute precisely the full distribution of test errors among interpolating classifiers. We find that test errors tend to concentrate around a small typical value $varepsilon*$, which deviates substantially from the test error of worst-case interpolating model. Our results show that the usual style of analysis in statistical learning theory may not be fine-grained enough to capture the good generalization performance observed in practice.
  arXiv  Detail & Related papers  (2020-06-22T21:12:31Z)
• Review of Probability Distributions for Modeling Count Data [0.0]
  Generalized linear models enable direct modeling of counts in a regression context. When counts contain only relative information, multinomial or Dirichlet-multinomial models can be more appropriate.
  arXiv  Detail & Related papers  (2020-01-10T18:28:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.