Bayesian Inference with Latent Hamiltonian Neural Networks

• URL: http://arxiv.org/abs/2208.06120v1
• Date: Fri, 12 Aug 2022 05:10:18 GMT
• Title: Bayesian Inference with Latent Hamiltonian Neural Networks
• Authors: Somayajulu L. N. Dhulipala, Yifeng Che, Michael D. Shields
• Abstract summary: Hamiltonian neural networks (HNNs) with Hamiltonian Monte Carlo (HMC) and No-U-Turn Sampler (NUTS) HNNs do not require numerical gradients of the target density during sampling. L-HNNs in NUTS with online error monitoring required 1--2 orders of magnitude fewer numerical gradients of the target density.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: When sampling for Bayesian inference, one popular approach is to use Hamiltonian Monte Carlo (HMC) and specifically the No-U-Turn Sampler (NUTS) which automatically decides the end time of the Hamiltonian trajectory. However, HMC and NUTS can require numerous numerical gradients of the target density, and can prove slow in practice. We propose Hamiltonian neural networks (HNNs) with HMC and NUTS for solving Bayesian inference problems. Once trained, HNNs do not require numerical gradients of the target density during sampling. Moreover, they satisfy important properties such as perfect time reversibility and Hamiltonian conservation, making them well-suited for use within HMC and NUTS because stationarity can be shown. We also propose an HNN extension called latent HNNs (L-HNNs), which are capable of predicting latent variable outputs. Compared to HNNs, L-HNNs offer improved expressivity and reduced integration errors. Finally, we employ L-HNNs in NUTS with an online error monitoring scheme to prevent sample degeneracy in regions of low probability density. We demonstrate L-HNNs in NUTS with online error monitoring on several examples involving complex, heavy-tailed, and high-local-curvature probability densities. Overall, L-HNNs in NUTS with online error monitoring satisfactorily inferred these probability densities. Compared to traditional NUTS, L-HNNs in NUTS with online error monitoring required 1--2 orders of magnitude fewer numerical gradients of the target density and improved the effective sample size (ESS) per gradient by an order of magnitude.

Related papers

• Scalable Mechanistic Neural Networks [52.28945097811129]
  We propose an enhanced neural network framework designed for scientific machine learning applications involving long temporal sequences. By reformulating the original Mechanistic Neural Network (MNN) we reduce the computational time and space complexities from cubic and quadratic with respect to the sequence length, respectively, to linear. Extensive experiments demonstrate that S-MNN matches the original MNN in precision while substantially reducing computational resources.
  arXiv  Detail & Related papers  (2024-10-08T14:27:28Z)
• Sparse Spiking Neural Network: Exploiting Heterogeneity in Timescales for Pruning Recurrent SNN [19.551319330414085]
  Spiking Neural Networks (RSNNs) have emerged as a computationally efficient and brain-inspired learning model. Traditionally, sparse SNNs are obtained by first training a dense and complex SNN for a target task. This paper presents a task-agnostic methodology for designing sparse RSNNs by pruning a large randomly model.
  arXiv  Detail & Related papers  (2024-03-06T02:36:15Z)
• Low Latency of object detection for spikng neural network [3.404826786562694]
  Spiking Neural Networks are well-suited for edge AI applications due to their binary spike nature. In this paper, we focus on generating highly accurate and low-latency SNNs specifically for object detection.
  arXiv  Detail & Related papers  (2023-09-27T10:26:19Z)
• An Automata-Theoretic Approach to Synthesizing Binarized Neural Networks [13.271286153792058]
  Quantized neural networks (QNNs) have been developed, with binarized neural networks (BNNs) restricted to binary values as a special case. This paper presents an automata-theoretic approach to synthesizing BNNs that meet designated properties.
  arXiv  Detail & Related papers  (2023-07-29T06:27:28Z)
• Benign Overfitting in Deep Neural Networks under Lazy Training [72.28294823115502]
  We show that when the data distribution is well-separated, DNNs can achieve Bayes-optimal test error for classification. Our results indicate that interpolating with smoother functions leads to better generalization.
  arXiv  Detail & Related papers  (2023-05-30T19:37:44Z)
• Physics-Informed Machine Learning of Dynamical Systems for Efficient Bayesian Inference [0.0]
  No-u-turn sampler (NUTS) is a widely adopted method for performing Bayesian inference. Hamiltonian neural networks (HNNs) are a noteworthy architecture. We propose the use of HNNs for performing Bayesian inference efficiently without requiring numerous posterior gradients.
  arXiv  Detail & Related papers  (2022-09-19T21:17:23Z)
• tinySNN: Towards Memory- and Energy-Efficient Spiking Neural Networks [14.916996986290902]
  Spiking Neural Network (SNN) models are typically favorable as they can offer higher accuracy. However, employing such models on the resource- and energy-constrained embedded platforms is inefficient. We present a tinySNN framework that optimize the memory and energy requirements of SNN processing.
  arXiv  Detail & Related papers  (2022-06-17T09:40:40Z)
• Learning Trajectories of Hamiltonian Systems with Neural Networks [81.38804205212425]
  We propose to enhance Hamiltonian neural networks with an estimation of a continuous-time trajectory of the modeled system. We demonstrate that the proposed integration scheme works well for HNNs, especially with low sampling rates, noisy and irregular observations.
  arXiv  Detail & Related papers  (2022-04-11T13:25:45Z)
• Comparative Analysis of Interval Reachability for Robust Implicit and Feedforward Neural Networks [64.23331120621118]
  We use interval reachability analysis to obtain robustness guarantees for implicit neural networks (INNs) INNs are a class of implicit learning models that use implicit equations as layers. We show that our approach performs at least as well as, and generally better than, applying state-of-the-art interval bound propagation methods to INNs.
  arXiv  Detail & Related papers  (2022-04-01T03:31:27Z)
• A Biased Graph Neural Network Sampler with Near-Optimal Regret [57.70126763759996]
  Graph neural networks (GNN) have emerged as a vehicle for applying deep network architectures to graph and relational data. In this paper, we build upon existing work and treat GNN neighbor sampling as a multi-armed bandit problem. We introduce a newly-designed reward function that introduces some degree of bias designed to reduce variance and avoid unstable, possibly-unbounded payouts.
  arXiv  Detail & Related papers  (2021-03-01T15:55:58Z)
• Toward Trainability of Quantum Neural Networks [87.04438831673063]
  Quantum Neural Networks (QNNs) have been proposed as generalizations of classical neural networks to achieve the quantum speed-up. Serious bottlenecks exist for training QNNs due to the vanishing with gradient rate exponential to the input qubit number. We show that QNNs with tree tensor and step controlled structures for the application of binary classification. Simulations show faster convergent rates and better accuracy compared to QNNs with random structures.
  arXiv  Detail & Related papers  (2020-11-12T08:32:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.