GIST: Gibbs self-tuning for locally adaptive Hamiltonian Monte Carlo

• URL: http://arxiv.org/abs/2404.15253v3
• Date: Thu, 03 Oct 2024 09:19:42 GMT
• Title: GIST: Gibbs self-tuning for locally adaptive Hamiltonian Monte Carlo
• Authors: Nawaf Bou-Rabee, Bob Carpenter, Milo Marsden,
• Abstract summary: We introduce a novel framework for constructing locally adaptive Hamiltonian Monte Carlo samplers by Gibbs sampling the algorithm's tuning parameters conditionally. For adaptively sampling path lengths, this framework -- which we call Gibbs self-tuning (GIST) -- encompasses randomized HMC, multinomial HMC, the No-U-Turn Sampler (NUTS) and the Apogee-to-Apogee Path Sampler as special cases.
• Score: 0.716879432974126
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce a novel and flexible framework for constructing locally adaptive Hamiltonian Monte Carlo (HMC) samplers by Gibbs sampling the algorithm's tuning parameters conditionally based on the position and momentum at each step. For adaptively sampling path lengths, this framework -- which we call Gibbs self-tuning (GIST) -- encompasses randomized HMC, multinomial HMC, the No-U-Turn Sampler (NUTS), and the Apogee-to-Apogee Path Sampler as special cases. The GIST framework is illustrated with a novel alternative to NUTS for locally adapting path lengths, evaluated with an exact Hamiltonian for a high-dimensional, ill-conditioned Gaussian measure and with the leapfrog integrator for a suite of diverse models.

Related papers

• Numerical Generalized Randomized Hamiltonian Monte Carlo for piecewise smooth target densities [0.0]
  Generalized Hamiltonian Monte Carlo processes for sampling continuous densities with discontinuous gradient and piecewise smooth targets are proposed. It is argued that the techniques lead to GRHMC processes that admit the desired target distribution as the invariant distribution in both scenarios.
  arXiv  Detail & Related papers  (2025-04-25T09:41:57Z)
• Bayesian Circular Regression with von Mises Quasi-Processes [57.88921637944379]
  In this work we explore a family of expressive and interpretable distributions over circle-valued random functions. For posterior inference, we introduce a new Stratonovich-like augmentation that lends itself to fast Gibbs sampling. We present experiments applying this model to the prediction of wind directions and the percentage of the running gait cycle as a function of joint angles.
  arXiv  Detail & Related papers  (2024-06-19T01:57:21Z)
• Gaussian Process Regression with Soft Inequality and Monotonicity Constraints [0.0]
  We introduce a new GP method that enforces the physical constraints in a probabilistic manner. This GP model is trained by the quantum-inspired Hamiltonian Monte Carlo (QHMC)
  arXiv  Detail & Related papers  (2024-04-03T17:09:25Z)
• Non-Gaussian Process Regression [0.0]
  We extend the GP framework into a new class of time-changed GPs that allow for straightforward modelling of heavy-tailed non-Gaussian behaviours. We present Markov chain Monte Carlo inference procedures for this model and demonstrate the potential benefits.
  arXiv  Detail & Related papers  (2022-09-07T13:08:22Z)
• Surrogate modeling for Bayesian optimization beyond a single Gaussian process [62.294228304646516]
  We propose a novel Bayesian surrogate model to balance exploration with exploitation of the search space. To endow function sampling with scalability, random feature-based kernel approximation is leveraged per GP model. To further establish convergence of the proposed EGP-TS to the global optimum, analysis is conducted based on the notion of Bayesian regret.
  arXiv  Detail & Related papers  (2022-05-27T16:43:10Z)
• B\'ezier Curve Gaussian Processes [8.11969931278838]
  This paper proposes a new probabilistic sequence model building on probabilistic B'ezier curves. Combined with a Mixture Density network, Bayesian conditional inference can be performed without the need for mean field variational approximation. The model is used for pedestrian trajectory prediction, where a generated prediction also serves as a GP prior.
  arXiv  Detail & Related papers  (2022-05-03T19:49:57Z)
• Wrapped Distributions on homogeneous Riemannian manifolds [58.720142291102135]
  Control over distributions' properties, such as parameters, symmetry and modality yield a family of flexible distributions. We empirically validate our approach by utilizing our proposed distributions within a variational autoencoder and a latent space network model.
  arXiv  Detail & Related papers  (2022-04-20T21:25:21Z)
• Non-Gaussian Gaussian Processes for Few-Shot Regression [71.33730039795921]
  We propose an invertible ODE-based mapping that operates on each component of the random variable vectors and shares the parameters across all of them. NGGPs outperform the competing state-of-the-art approaches on a diversified set of benchmarks and applications.
  arXiv  Detail & Related papers  (2021-10-26T10:45:25Z)
• Deterministic Gibbs Sampling via Ordinary Differential Equations [77.42706423573573]
  This paper presents a general construction of deterministic measure-preserving dynamics using autonomous ODEs and tools from differential geometry. We show how Hybrid Monte Carlo and other deterministic samplers follow as special cases of our theory.
  arXiv  Detail & Related papers  (2021-06-18T15:36:09Z)
• Invertible Flow Non Equilibrium sampling [10.068677972360318]
  Invertible Flow Non Equilibrium Sampling (InFine) InFine constructs unbiased estimators of expectations and in particular of normalizing constants. Can be used to construct an Evidence Lower Bound (ELBO) leading to a new class of Variational AutoEncoders (VAE)
  arXiv  Detail & Related papers  (2021-03-17T09:09:06Z)
• Generative Neural Samplers for the Quantum Heisenberg Chain [0.3655021726150368]
  Generative neural samplers offer a complementary approach to Monte Carlo methods for problems in statistical physics and quantum field theory. This work tests the ability of generative neural samplers to estimate observables for real-world low-dimensional spin systems.
  arXiv  Detail & Related papers  (2020-12-18T14:28:13Z)
• Gaussian Process-based Min-norm Stabilizing Controller for Control-Affine Systems with Uncertain Input Effects and Dynamics [90.81186513537777]
  We propose a novel compound kernel that captures the control-affine nature of the problem. We show that this resulting optimization problem is convex, and we call it Gaussian Process-based Control Lyapunov Function Second-Order Cone Program (GP-CLF-SOCP)
  arXiv  Detail & Related papers  (2020-11-14T01:27:32Z)
• Sinkhorn Natural Gradient for Generative Models [125.89871274202439]
  We propose a novel Sinkhorn Natural Gradient (SiNG) algorithm which acts as a steepest descent method on the probability space endowed with the Sinkhorn divergence. We show that the Sinkhorn information matrix (SIM), a key component of SiNG, has an explicit expression and can be evaluated accurately in complexity that scales logarithmically. In our experiments, we quantitatively compare SiNG with state-of-the-art SGD-type solvers on generative tasks to demonstrate its efficiency and efficacy of our method.
  arXiv  Detail & Related papers  (2020-11-09T02:51:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.