Hamiltonian Monte Carlo Inference of Marginalized Linear Mixed-Effects Models

• URL: http://arxiv.org/abs/2410.24079v2
• Date: Thu, 21 Nov 2024 17:33:18 GMT
• Title: Hamiltonian Monte Carlo Inference of Marginalized Linear Mixed-Effects Models
• Authors: Jinlin Lai, Justin Domke, Daniel Sheldon,
• Abstract summary: We develop an algorithm to easily marginalize random effects in linear mixed-effects models. A naive approach introduces cubic time operations within an inference algorithm like Hamiltonian Monte Carlo (HMC) We show that marginalization is always beneficial when applicable and highlight improvements in various models.
• Score: 26.534682620182707
• License:
• Abstract: Bayesian reasoning in linear mixed-effects models (LMMs) is challenging and often requires advanced sampling techniques like Markov chain Monte Carlo (MCMC). A common approach is to write the model in a probabilistic programming language and then sample via Hamiltonian Monte Carlo (HMC). However, there are many ways a user can transform a model that make inference more or less efficient. In particular, marginalizing some variables can greatly improve inference but is difficult for users to do manually. We develop an algorithm to easily marginalize random effects in LMMs. A naive approach introduces cubic time operations within an inference algorithm like HMC, but we reduce the running time to linear using fast linear algebra techniques. We show that marginalization is always beneficial when applicable and highlight improvements in various models, especially ones from cognitive sciences.

Related papers

• Promises and Pitfalls of Generative Masked Language Modeling: Theoretical Framework and Practical Guidelines [74.42485647685272]
  We focus on Generative Masked Language Models (GMLMs) We train a model to fit conditional probabilities of the data distribution via masking, which are subsequently used as inputs to a Markov Chain to draw samples from the model. We adapt the T5 model for iteratively-refined parallel decoding, achieving 2-3x speedup in machine translation with minimal sacrifice in quality.
  arXiv  Detail & Related papers  (2024-07-22T18:00:00Z)
• Online Variational Sequential Monte Carlo [49.97673761305336]
  We build upon the variational sequential Monte Carlo (VSMC) method, which provides computationally efficient and accurate model parameter estimation and Bayesian latent-state inference. Online VSMC is capable of performing efficiently, entirely on-the-fly, both parameter estimation and particle proposal adaptation.
  arXiv  Detail & Related papers  (2023-12-19T21:45:38Z)
• Probabilistic Unrolling: Scalable, Inverse-Free Maximum Likelihood Estimation for Latent Gaussian Models [69.22568644711113]
  We introduce probabilistic unrolling, a method that combines Monte Carlo sampling with iterative linear solvers to circumvent matrix inversions. Our theoretical analyses reveal that unrolling and backpropagation through the iterations of the solver can accelerate gradient estimation for maximum likelihood estimation. In experiments on simulated and real data, we demonstrate that probabilistic unrolling learns latent Gaussian models up to an order of magnitude faster than gradient EM, with minimal losses in model performance.
  arXiv  Detail & Related papers  (2023-06-05T21:08:34Z)
• Automatically Marginalized MCMC in Probabilistic Programming [12.421267523795114]
  Hamiltonian Monte Carlo (HMC) is a powerful algorithm to sample latent variables from Bayesian models. We propose to use automatic marginalization as part of the sampling process using HMC in a graphical model extracted from a PPL.
  arXiv  Detail & Related papers  (2023-02-01T16:34:07Z)
• Compound Batch Normalization for Long-tailed Image Classification [77.42829178064807]
  We propose a compound batch normalization method based on a Gaussian mixture. It can model the feature space more comprehensively and reduce the dominance of head classes. The proposed method outperforms existing methods on long-tailed image classification.
  arXiv  Detail & Related papers  (2022-12-02T07:31:39Z)
• Low-variance estimation in the Plackett-Luce model via quasi-Monte Carlo sampling [58.14878401145309]
  We develop a novel approach to producing more sample-efficient estimators of expectations in the PL model. We illustrate our findings both theoretically and empirically using real-world recommendation data from Amazon Music and the Yahoo learning-to-rank challenge.
  arXiv  Detail & Related papers  (2022-05-12T11:15:47Z)
• A fast asynchronous MCMC sampler for sparse Bayesian inference [10.535140830570256]
  We propose a very fast approximate Markov Chain Monte Carlo (MCMC) sampling framework that is applicable to a large class of sparse Bayesian inference problems. We show that in high-dimensional linear regression problems, the Markov chain generated by the proposed algorithm admits an invariant distribution that recovers correctly the main signal.
  arXiv  Detail & Related papers  (2021-08-14T02:20:49Z)
• Bayesian Inference in High-Dimensional Time-Serieswith the Orthogonal Stochastic Linear Mixing Model [2.7909426811685893]
  Many modern time-series datasets contain large numbers of output response variables sampled for prolonged periods of time. In this paper, we propose a new Markov chain Monte Carlo framework for the analysis of diverse, large-scale time-series datasets.
  arXiv  Detail & Related papers  (2021-06-25T01:12:54Z)
• Scaling Bayesian inference of mixed multinomial logit models to very large datasets [9.442139459221785]
  We propose an Amortized Variational Inference approach that leverages backpropagation, automatic differentiation and GPU-accelerated computation. We show how normalizing flows can be used to increase the flexibility of the variational posterior approximations.
  arXiv  Detail & Related papers  (2020-04-11T15:30:47Z)
• Learning Gaussian Graphical Models via Multiplicative Weights [54.252053139374205]
  We adapt an algorithm of Klivans and Meka based on the method of multiplicative weight updates. The algorithm enjoys a sample complexity bound that is qualitatively similar to others in the literature. It has a low runtime $O(mp2)$ in the case of $m$ samples and $p$ nodes, and can trivially be implemented in an online manner.
  arXiv  Detail & Related papers  (2020-02-20T10:50:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.