Antithetic Riemannian Manifold And Quantum-Inspired Hamiltonian Monte Carlo

• URL: http://arxiv.org/abs/2107.02070v1
• Date: Mon, 5 Jul 2021 15:03:07 GMT
• Title: Antithetic Riemannian Manifold And Quantum-Inspired Hamiltonian Monte Carlo
• Authors: Wilson Tsakane Mongwe, Rendani Mbuvha, Tshilidzi Marwala
• Abstract summary: We present new algorithms which are antithetic versions of Hamiltonian Monte Carlo and Quantum-Inspired Hamiltonian Monte Carlo. Adding antithetic sampling to Hamiltonian Monte Carlo has been previously shown to produce higher effective sample rates compared to vanilla Hamiltonian Monte Carlo. The analysis is performed on jump diffusion process using real world financial market data, as well as on real world benchmark classification tasks using Bayesian logistic regression.
• Score: 3.686886131767452
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Markov Chain Monte Carlo inference of target posterior distributions in machine learning is predominately conducted via Hamiltonian Monte Carlo and its variants. This is due to Hamiltonian Monte Carlo based samplers ability to suppress random-walk behaviour. As with other Markov Chain Monte Carlo methods, Hamiltonian Monte Carlo produces auto-correlated samples which results in high variance in the estimators, and low effective sample size rates in the generated samples. Adding antithetic sampling to Hamiltonian Monte Carlo has been previously shown to produce higher effective sample rates compared to vanilla Hamiltonian Monte Carlo. In this paper, we present new algorithms which are antithetic versions of Riemannian Manifold Hamiltonian Monte Carlo and Quantum-Inspired Hamiltonian Monte Carlo. The Riemannian Manifold Hamiltonian Monte Carlo algorithm improves on Hamiltonian Monte Carlo by taking into account the local geometry of the target, which is beneficial for target densities that may exhibit strong correlations in the parameters. Quantum-Inspired Hamiltonian Monte Carlo is based on quantum particles that can have random mass. Quantum-Inspired Hamiltonian Monte Carlo uses a random mass matrix which results in better sampling than Hamiltonian Monte Carlo on spiky and multi-modal distributions such as jump diffusion processes. The analysis is performed on jump diffusion process using real world financial market data, as well as on real world benchmark classification tasks using Bayesian logistic regression.

Related papers

• von Mises Quasi-Processes for Bayesian Circular Regression [57.88921637944379]
  We explore a family of expressive and interpretable distributions over circle-valued random functions. The resulting probability model has connections with continuous spin models in statistical physics. For posterior inference, we introduce a new Stratonovich-like augmentation that lends itself to fast Markov Chain Monte Carlo sampling.
  arXiv  Detail & Related papers  (2024-06-19T01:57:21Z)
• Combining Normalizing Flows and Quasi-Monte Carlo [0.0]
  Recent advances in machine learning have led to the development of new methods for enhancing Monte Carlo methods. We demonstrate through numerical experiments that this combination can lead to an estimator with significantly lower variance than if the flow was sampled with a classic Monte Carlo.
  arXiv  Detail & Related papers  (2024-01-11T14:17:06Z)
• Randomized Runge-Kutta-Nyström Methods for Unadjusted Hamiltonian and Kinetic Langevin Monte Carlo [0.0]
  We introduce $5/2$- and $7/2$-accurate randomized Runge-Kutta-Nystr"om methods for approximating Hamiltonian flows within non-reversible Markov chain Monte Carlo samplers. The numerical experiments demonstrate the superior efficiency of the proposed unadjusted samplers on a variety of well-behaved, high-dimensional target distributions.
  arXiv  Detail & Related papers  (2023-10-11T11:35:12Z)
• Stochastic automatic differentiation for Monte Carlo processes [1.1279808969568252]
  We consider the extension of Automatic Differentiation (AD) techniques to Monte Carlo process. We show that the Hamiltonian approach can be understood as a change of variables of the reweighting approach.
  arXiv  Detail & Related papers  (2023-07-28T08:59:01Z)
• Dilute neutron star matter from neural-network quantum states [58.720142291102135]
  Low-density neutron matter is characterized by the formation of Cooper pairs and the onset of superfluidity. We model this density regime by capitalizing on the expressivity of the hidden-nucleon neural-network quantum states combined with variational Monte Carlo and reconfiguration techniques.
  arXiv  Detail & Related papers  (2022-12-08T17:55:25Z)
• Low-variance estimation in the Plackett-Luce model via quasi-Monte Carlo sampling [58.14878401145309]
  We develop a novel approach to producing more sample-efficient estimators of expectations in the PL model. We illustrate our findings both theoretically and empirically using real-world recommendation data from Amazon Music and the Yahoo learning-to-rank challenge.
  arXiv  Detail & Related papers  (2022-05-12T11:15:47Z)
• An Introduction to Hamiltonian Monte Carlo Method for Sampling [26.555110725656963]
  Hamiltonian Monte Carlo (HMC) is a Hamiltonian dynamics-inspired algorithm for sampling from a Gibbs density $pi(x) propto e-f(x)$. We show that idealized HMC preserves $pi$ and we establish its convergence when $f$ is strongly convex and smooth.
  arXiv  Detail & Related papers  (2021-08-27T03:28:20Z)
• Deterministic Gibbs Sampling via Ordinary Differential Equations [77.42706423573573]
  This paper presents a general construction of deterministic measure-preserving dynamics using autonomous ODEs and tools from differential geometry. We show how Hybrid Monte Carlo and other deterministic samplers follow as special cases of our theory.
  arXiv  Detail & Related papers  (2021-06-18T15:36:09Z)
• Annealed Flow Transport Monte Carlo [91.20263039913912]
  Annealed Flow Transport (AFT) builds upon Annealed Importance Sampling (AIS) and Sequential Monte Carlo (SMC) AFT relies on NF which is learned sequentially to push particles towards the successive targets. We show that a continuous-time scaling limit of the population version of AFT is given by a Feynman--Kac measure.
  arXiv  Detail & Related papers  (2021-02-15T12:05:56Z)
• Sampling in Combinatorial Spaces with SurVAE Flow Augmented MCMC [83.48593305367523]
  Hybrid Monte Carlo is a powerful Markov Chain Monte Carlo method for sampling from complex continuous distributions. We introduce a new approach based on augmenting Monte Carlo methods with SurVAE Flows to sample from discrete distributions. We demonstrate the efficacy of our algorithm on a range of examples from statistics, computational physics and machine learning, and observe improvements compared to alternative algorithms.
  arXiv  Detail & Related papers  (2021-02-04T02:21:08Z)
• Generative Neural Samplers for the Quantum Heisenberg Chain [0.3655021726150368]
  Generative neural samplers offer a complementary approach to Monte Carlo methods for problems in statistical physics and quantum field theory. This work tests the ability of generative neural samplers to estimate observables for real-world low-dimensional spin systems.
  arXiv  Detail & Related papers  (2020-12-18T14:28:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.