On the mapping between Hopfield networks and Restricted Boltzmann Machines

• URL: http://arxiv.org/abs/2101.11744v2
• Date: Sat, 6 Mar 2021 02:08:12 GMT
• Title: On the mapping between Hopfield networks and Restricted Boltzmann Machines
• Authors: Matthew Smart, Anton Zilman
• Abstract summary: We show an exact mapping between Hopfield networks (HNs) and Restricted Boltzmann Machines (RBMs) We outline the conditions under which the reverse mapping exists, and conduct experiments on the MNIST dataset. We discuss extensions, the potential importance of this correspondence for the training of RBMs, and for understanding the performance of deep architectures which utilize RBMs.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Hopfield networks (HNs) and Restricted Boltzmann Machines (RBMs) are two important models at the interface of statistical physics, machine learning, and neuroscience. Recently, there has been interest in the relationship between HNs and RBMs, due to their similarity under the statistical mechanics formalism. An exact mapping between HNs and RBMs has been previously noted for the special case of orthogonal (uncorrelated) encoded patterns. We present here an exact mapping in the case of correlated pattern HNs, which are more broadly applicable to existing datasets. Specifically, we show that any HN with $N$ binary variables and $p<N$ arbitrary binary patterns can be transformed into an RBM with $N$ binary visible variables and $p$ gaussian hidden variables. We outline the conditions under which the reverse mapping exists, and conduct experiments on the MNIST dataset which suggest the mapping provides a useful initialization to the RBM weights. We discuss extensions, the potential importance of this correspondence for the training of RBMs, and for understanding the performance of deep architectures which utilize RBMs.

Related papers

• Scalable Weibull Graph Attention Autoencoder for Modeling Document Networks [50.42343781348247]
  We develop a graph Poisson factor analysis (GPFA) which provides analytic conditional posteriors to improve the inference accuracy. We also extend GPFA to a multi-stochastic-layer version named graph Poisson gamma belief network (GPGBN) to capture the hierarchical document relationships at multiple semantic levels. Our models can extract high-quality hierarchical latent document representations and achieve promising performance on various graph analytic tasks.
  arXiv  Detail & Related papers  (2024-10-13T02:22:14Z)
• Learning Restricted Boltzmann Machines with greedy quantum search [2.98017021422101]
  We extend scope to the quantum computing domain and propose corresponding quantum algorithms for this problem. Our study demonstrates that the proposed quantum algorithms yield a speedup compared to the classical algorithms for learning the structure of these two classes of RBMs.
  arXiv  Detail & Related papers  (2023-09-25T14:56:30Z)
• DA-VEGAN: Differentiably Augmenting VAE-GAN for microstructure reconstruction from extremely small data sets [110.60233593474796]
  DA-VEGAN is a model with two central innovations. A $beta$-variational autoencoder is incorporated into a hybrid GAN architecture. A custom differentiable data augmentation scheme is developed specifically for this architecture.
  arXiv  Detail & Related papers  (2023-02-17T08:49:09Z)
• Tree Mover's Distance: Bridging Graph Metrics and Stability of Graph Neural Networks [54.225220638606814]
  We propose a pseudometric for attributed graphs, the Tree Mover's Distance (TMD), and study its relation to generalization. First, we show that TMD captures properties relevant to graph classification; a simple TMD-SVM performs competitively with standard GNNs. Second, we relate TMD to generalization of GNNs under distribution shifts, and show that it correlates well with performance drop under such shifts.
  arXiv  Detail & Related papers  (2022-10-04T21:03:52Z)
• Batch-Ensemble Stochastic Neural Networks for Out-of-Distribution Detection [55.028065567756066]
  Out-of-distribution (OOD) detection has recently received much attention from the machine learning community due to its importance in deploying machine learning models in real-world applications. In this paper we propose an uncertainty quantification approach by modelling the distribution of features. We incorporate an efficient ensemble mechanism, namely batch-ensemble, to construct the batch-ensemble neural networks (BE-SNNs) and overcome the feature collapse problem. We show that BE-SNNs yield superior performance on several OOD benchmarks, such as the Two-Moons dataset, the FashionMNIST vs MNIST dataset, FashionM
  arXiv  Detail & Related papers  (2022-06-26T16:00:22Z)
• Restricted Boltzmann Machine and Deep Belief Network: Tutorial and Survey [5.967999555890417]
  This tutorial and survey paper is on Boltzmann Machine (BM), Restricted Boltzmann Machine (RBM), and Deep Belief Network (DBN) We start with the required background on probabilistic graphical models, Markov random field, Gibbs sampling, statistical physics, Ising model, and the Hopfield network. The conditional distributions of visible and hidden variables, Gibbs sampling in RBM for generating variables, training BM and RBM by maximum likelihood estimation, and contrastive divergence are explained.
  arXiv  Detail & Related papers  (2021-07-26T23:59:12Z)
• Barriers and Dynamical Paths in Alternating Gibbs Sampling of Restricted Boltzmann Machines [0.0]
  We study the performance of Alternating Gibbs Sampling (AGS) on several analytically tractable models. We show that standard AGS is not more efficient than classical Metropolis-Hastings (MH) sampling of the effective energy landscape. We illustrate our findings on three datasets: Bars and Stripes and MNIST, well known in machine learning, and the so-called Lattice Proteins.
  arXiv  Detail & Related papers  (2021-07-13T12:07:56Z)
• Restricted Boltzmann Machine, recent advances and mean-field theory [0.8702432681310401]
  Review deals with Restricted Boltzmann Machine (RBM) under the light of statistical physics. RBM is a classical family of Machine learning (ML) models which played a central role in the development of deep learning.
  arXiv  Detail & Related papers  (2020-11-23T10:08:53Z)
• Exact representations of many body interactions with RBM neural networks [77.34726150561087]
  We exploit the representation power of RBMs to provide an exact decomposition of many-body contact interactions into one-body operators. This construction generalizes the well known Hirsch's transform used for the Hubbard model to more complicated theories such as Pionless EFT in nuclear physics.
  arXiv  Detail & Related papers  (2020-05-07T15:59:29Z)
• On the Difference Between the Information Bottleneck and the Deep Information Bottleneck [81.89141311906552]
  We revisit the Deep Variational Information Bottleneck and the assumptions needed for its derivation. We show how to circumvent this limitation by optimising a lower bound for $I(T;Y)$ for which only the latter Markov chain has to be satisfied.
  arXiv  Detail & Related papers  (2019-12-31T18:31:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.