Fast Riemannian-manifold Hamiltonian Monte Carlo for hierarchical Gaussian-process models

• URL: http://arxiv.org/abs/2511.06407v1
• Date: Sun, 09 Nov 2025 14:44:13 GMT
• Title: Fast Riemannian-manifold Hamiltonian Monte Carlo for hierarchical Gaussian-process models
• Authors: Takashi Hayakawa, Satoshi Asai,
• Abstract summary: We show that, compared with the slow inference achieved with existing program libraries, the performance can be drastically improved.<n>We demonstrate that RMHMC effectively samples from the posterior, allowing the calculation of model evidence.<n>We highlight the need to develop a customisable library set that allows users to incorporate dynamically programmed objects.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Hierarchical Bayesian models based on Gaussian processes are considered useful for describing complex nonlinear statistical dependencies among variables in real-world data. However, effective Monte Carlo algorithms for inference with these models have not yet been established, except for several simple cases. In this study, we show that, compared with the slow inference achieved with existing program libraries, the performance of Riemannian-manifold Hamiltonian Monte Carlo (RMHMC) can be drastically improved by optimising the computation order according to the model structure and dynamically programming the eigendecomposition. This improvement cannot be achieved when using an existing library based on a naive automatic differentiator. We numerically demonstrate that RMHMC effectively samples from the posterior, allowing the calculation of model evidence, in a Bayesian logistic regression on simulated data and in the estimation of propensity functions for the American national medical expenditure data using several Bayesian multiple-kernel models. These results lay a foundation for implementing effective Monte Carlo algorithms for analysing real-world data with Gaussian processes, and highlight the need to develop a customisable library set that allows users to incorporate dynamically programmed objects and finely optimises the mode of automatic differentiation depending on the model structure.

Related papers

• Efficient Stochastic Optimisation via Sequential Monte Carlo [0.5599792629509229]
  We develop sequential Monte Carlo samplers for optimisation of functions with intractable gradients.<n>Our approach replaces expensive inner sampling methods with efficient SMC approximations, which can result in significant computational gains.<n>We demonstrate the effectiveness of our approach on the reward-tuning of energy-based models within various settings.
  arXiv  Detail & Related papers  (2026-01-29T17:13:25Z)
• FBMS: An R Package for Flexible Bayesian Model Selection and Model Averaging [14.487258585834374]
  The FBMS package implements an efficient Mode Jumping Markov Chain Monte Carlo (MJMCMC) algorithm.<n>Within this framework, the algorithm maintains and updates populations of transformed features, computes their posterior probabilities, and evaluates the posteriors of models constructed from them.<n>We demonstrate the effective use of FBMS for both inferential and predictive modeling in Gaussian regression, focusing on different instances of the BGNLM class of models.
  arXiv  Detail & Related papers  (2025-08-31T09:04:01Z)
• Bayesian Data Sketching for Varying Coefficient Regression Models [1.6727186769396276]
  We introduce Bayesian data sketching for varying coefficient models to obviate computational challenges presented by large sample sizes.<n>Our approach distinguishes itself from several existing methods for analyzing large functional data.<n>Well-established methods and algorithms for varying coefficient regression models can be applied to the compressed data.
  arXiv  Detail & Related papers  (2025-05-30T22:09:06Z)
• Scalable Inference for Bayesian Multinomial Logistic-Normal Dynamic Linear Models [0.5735035463793009]
  This article develops an efficient and accurate approach to posterior state estimation, called $textitFenrir$.<n>Our experiments suggest that Fenrir can be three orders of magnitude more efficient than Stan.<n>Our methods are made available to the community as a user-friendly software library written in C++ with an R interface.
  arXiv  Detail & Related papers  (2024-10-07T23:20:14Z)
• Fusion of Gaussian Processes Predictions with Monte Carlo Sampling [61.31380086717422]
  In science and engineering, we often work with models designed for accurate prediction of variables of interest. Recognizing that these models are approximations of reality, it becomes desirable to apply multiple models to the same data and integrate their outcomes.
  arXiv  Detail & Related papers  (2024-03-03T04:21:21Z)
• Online Variational Sequential Monte Carlo [49.97673761305336]
  We build upon the variational sequential Monte Carlo (VSMC) method, which provides computationally efficient and accurate model parameter estimation and Bayesian latent-state inference. Online VSMC is capable of performing efficiently, entirely on-the-fly, both parameter estimation and particle proposal adaptation.
  arXiv  Detail & Related papers  (2023-12-19T21:45:38Z)
• Probabilistic Unrolling: Scalable, Inverse-Free Maximum Likelihood Estimation for Latent Gaussian Models [69.22568644711113]
  We introduce probabilistic unrolling, a method that combines Monte Carlo sampling with iterative linear solvers to circumvent matrix inversions. Our theoretical analyses reveal that unrolling and backpropagation through the iterations of the solver can accelerate gradient estimation for maximum likelihood estimation. In experiments on simulated and real data, we demonstrate that probabilistic unrolling learns latent Gaussian models up to an order of magnitude faster than gradient EM, with minimal losses in model performance.
  arXiv  Detail & Related papers  (2023-06-05T21:08:34Z)
• Adaptive LASSO estimation for functional hidden dynamic geostatistical model [69.10717733870575]
  We propose a novel model selection algorithm based on a penalized maximum likelihood estimator (PMLE) for functional hiddenstatistical models (f-HD) The algorithm is based on iterative optimisation and uses an adaptive least absolute shrinkage and selector operator (GMSOLAS) penalty function, wherein the weights are obtained by the unpenalised f-HD maximum-likelihood estimators.
  arXiv  Detail & Related papers  (2022-08-10T19:17:45Z)
• Low-variance estimation in the Plackett-Luce model via quasi-Monte Carlo sampling [58.14878401145309]
  We develop a novel approach to producing more sample-efficient estimators of expectations in the PL model. We illustrate our findings both theoretically and empirically using real-world recommendation data from Amazon Music and the Yahoo learning-to-rank challenge.
  arXiv  Detail & Related papers  (2022-05-12T11:15:47Z)
• Spike-and-Slab Generalized Additive Models and Scalable Algorithms for High-Dimensional Data [0.0]
  We propose hierarchical generalized additive models (GAMs) to accommodate high-dimensional data. We consider the smoothing penalty for proper shrinkage of curve and separation of smoothing function linear and nonlinear spaces. Two and deterministic algorithms, EM-Coordinate Descent and EM-Iterative Weighted Least Squares, are developed for different utilities.
  arXiv  Detail & Related papers  (2021-10-27T14:11:13Z)
• Goal-directed Generation of Discrete Structures with Conditional Generative Models [85.51463588099556]
  We introduce a novel approach to directly optimize a reinforcement learning objective, maximizing an expected reward. We test our methodology on two tasks: generating molecules with user-defined properties and identifying short python expressions which evaluate to a given target value.
  arXiv  Detail & Related papers  (2020-10-05T20:03:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.