Hamiltonian Adaptive Importance Sampling

• URL: http://arxiv.org/abs/2209.13716v1
• Date: Tue, 27 Sep 2022 22:00:07 GMT
• Title: Hamiltonian Adaptive Importance Sampling
• Authors: Ali Mousavi, Reza Monsefi, and V\'ictor Elvira
• Abstract summary: We introduce the novel Hamiltonian adaptive importance sampling (HAIS) method. HAIS implements a two-step adaptive process with parallel HMC chains that cooperate at each iteration. It achieves a significant performance improvement in high-dimensional problems w.r.t. state-of-the-art algorithms.
• Score: 9.937177877192198
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Importance sampling (IS) is a powerful Monte Carlo (MC) methodology for approximating integrals, for instance in the context of Bayesian inference. In IS, the samples are simulated from the so-called proposal distribution, and the choice of this proposal is key for achieving a high performance. In adaptive IS (AIS) methods, a set of proposals is iteratively improved. AIS is a relevant and timely methodology although many limitations remain yet to be overcome, e.g., the curse of dimensionality in high-dimensional and multi-modal problems. Moreover, the Hamiltonian Monte Carlo (HMC) algorithm has become increasingly popular in machine learning and statistics. HMC has several appealing features such as its exploratory behavior, especially in high-dimensional targets, when other methods suffer. In this paper, we introduce the novel Hamiltonian adaptive importance sampling (HAIS) method. HAIS implements a two-step adaptive process with parallel HMC chains that cooperate at each iteration. The proposed HAIS efficiently adapts a population of proposals, extracting the advantages of HMC. HAIS can be understood as a particular instance of the generic layered AIS family with an additional resampling step. HAIS achieves a significant performance improvement in high-dimensional problems w.r.t. state-of-the-art algorithms. We discuss the statistical properties of HAIS and show its high performance in two challenging examples.

Related papers

• The Stochastic Conjugate Subgradient Algorithm For Kernel Support Vector Machines [1.738375118265695]
  This paper proposes an innovative method specifically designed for kernel support vector machines (SVMs) It not only achieves faster iteration per iteration but also exhibits enhanced convergence when compared to conventional SFO techniques. Our experimental results demonstrate that the proposed algorithm not only maintains but potentially exceeds the scalability of SFO methods.
  arXiv  Detail & Related papers  (2024-07-30T17:03:19Z)
• Efficient Multi-agent Reinforcement Learning by Planning [33.51282615335009]
  Multi-agent reinforcement learning (MARL) algorithms have accomplished remarkable breakthroughs in solving large-scale decision-making tasks. Most existing MARL algorithms are model-free, limiting sample efficiency and hindering their applicability in more challenging scenarios. We propose the MAZero algorithm, which combines a centralized model with Monte Carlo Tree Search (MCTS) for policy search.
  arXiv  Detail & Related papers  (2024-05-20T04:36:02Z)
• Multi-fidelity Hamiltonian Monte Carlo [1.86413150130483]
  We propose a novel two-stage Hamiltonian Monte Carlo algorithm with a surrogate model. The accepted probability is computed in the first stage via a standard HMC proposal. If the proposal is accepted, the posterior is evaluated in the second stage using the high-fidelity numerical solver.
  arXiv  Detail & Related papers  (2024-05-08T13:03:55Z)
• Maximize to Explore: One Objective Function Fusing Estimation, Planning, and Exploration [87.53543137162488]
  We propose an easy-to-implement online reinforcement learning (online RL) framework called textttMEX. textttMEX integrates estimation and planning components while balancing exploration exploitation automatically. It can outperform baselines by a stable margin in various MuJoCo environments with sparse rewards.
  arXiv  Detail & Related papers  (2023-05-29T17:25:26Z)
• Numerical Methods for Convex Multistage Stochastic Optimization [86.45244607927732]
  We focus on optimisation programming (SP), Optimal Control (SOC) and Decision Processes (MDP) Recent progress in solving convex multistage Markov problems is based on cutting planes approximations of the cost-to-go functions of dynamic programming equations. Cutting plane type methods can handle multistage problems with a large number of stages, but a relatively smaller number of state (decision) variables.
  arXiv  Detail & Related papers  (2023-03-28T01:30:40Z)
• Estimating Average Treatment Effects with Support Vector Machines [77.34726150561087]
  Support vector machine (SVM) is one of the most popular classification algorithms in the machine learning literature. We adapt SVM as a kernel-based weighting procedure that minimizes the maximum mean discrepancy between the treatment and control groups. We characterize the bias of causal effect estimation arising from this trade-off, connecting the proposed SVM procedure to the existing kernel balancing methods.
  arXiv  Detail & Related papers  (2021-02-23T20:22:56Z)
• Spatial Monte Carlo Integration with Annealed Importance Sampling [0.45687771576879593]
  A new method is proposed to evaluate the expectations on Ising models combining AIS and SMCI. The proposed method performs efficiently in both high- and low-temperature regions.
  arXiv  Detail & Related papers  (2020-12-21T09:26:40Z)
• Adaptive Sampling for Best Policy Identification in Markov Decision Processes [79.4957965474334]
  We investigate the problem of best-policy identification in discounted Markov Decision (MDPs) when the learner has access to a generative model. The advantages of state-of-the-art algorithms are discussed and illustrated.
  arXiv  Detail & Related papers  (2020-09-28T15:22:24Z)
• Differentiable Expected Hypervolume Improvement for Parallel Multi-Objective Bayesian Optimization [11.956059322407437]
  We leverage recent advances in programming models and hardware acceleration for multi-objective BO using Expected Hyper Improvement (EHVI) We derive a novel formulation of q-Expected Hyper Improvement (qEHVI), an acquisition function that extends EHVI to the parallel, constrained evaluation setting. Our empirical evaluation demonstrates that qEHVI is computationally tractable in many practical scenarios and outperforms state-of-the-art multi-objective BO algorithms at a fraction of their wall time.
  arXiv  Detail & Related papers  (2020-06-09T06:57:47Z)
• Improving Sampling Accuracy of Stochastic Gradient MCMC Methods via Non-uniform Subsampling of Gradients [54.90670513852325]
  We propose a non-uniform subsampling scheme to improve the sampling accuracy. EWSG is designed so that a non-uniform gradient-MCMC method mimics the statistical behavior of a batch-gradient-MCMC method. In our practical implementation of EWSG, the non-uniform subsampling is performed efficiently via a Metropolis-Hastings chain on the data index.
  arXiv  Detail & Related papers  (2020-02-20T18:56:18Z)
• Supervised Hyperalignment for multi-subject fMRI data alignment [81.8694682249097]
  This paper proposes a Supervised Hyperalignment (SHA) method to ensure better functional alignment for MVP analysis. Experiments on multi-subject datasets demonstrate that SHA method achieves up to 19% better performance for multi-class problems.
  arXiv  Detail & Related papers  (2020-01-09T09:17:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.