Universal representation by Boltzmann machines with Regularised Axons

• URL: http://arxiv.org/abs/2310.14395v2
• Date: Thu, 30 Nov 2023 23:14:44 GMT
• Title: Universal representation by Boltzmann machines with Regularised Axons
• Authors: Przemys{\l}aw R. Grzybowski, Antoni Jankiewicz, Eloy Pi\~nol, David Cirauqui, Dorota H. Grzybowska, Pawe{\l} M. Petrykowski, Miguel \'Angel Garc\'ia-March, Maciej Lewenstein, Gorka Mu\~noz-Gil, Alejandro Pozas-Kerstjens
• Abstract summary: We show that regularised Boltzmann machines preserve the ability to represent arbitrary distributions. We also show that regularised Boltzmann machines can store exponentially many arbitrarily correlated visible patterns with perfect retrieval.
• Score: 34.337412054122076
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: It is widely known that Boltzmann machines are capable of representing arbitrary probability distributions over the values of their visible neurons, given enough hidden ones. However, sampling -- and thus training -- these models can be numerically hard. Recently we proposed a regularisation of the connections of Boltzmann machines, in order to control the energy landscape of the model, paving a way for efficient sampling and training. Here we formally prove that such regularised Boltzmann machines preserve the ability to represent arbitrary distributions. This is in conjunction with controlling the number of energy local minima, thus enabling easy \emph{guided} sampling and training. Furthermore, we explicitly show that regularised Boltzmann machines can store exponentially many arbitrarily correlated visible patterns with perfect retrieval, and we connect them to the Dense Associative Memory networks.

Related papers

• Scalable Equilibrium Sampling with Sequential Boltzmann Generators [60.00515282300297]
  We extend the Boltzmann generator framework and introduce Sequential Boltzmann generators with two key improvements. The first is a highly efficient non-equivariant Transformer-based normalizing flow operating directly on all-atom Cartesian coordinates. We demonstrate the first equilibrium sampling in Cartesian coordinates of tri, tetra, and hexapeptides that were so far intractable for prior Boltzmann generators.
  arXiv  Detail & Related papers  (2025-02-25T18:59:13Z)
• Representing Arbitrary Ground States of Toric Code by Restricted Boltzmann Machine [0.23408308015481666]
  We analyze the representability of toric code ground states by Restricted Boltzmann Machine with only local connections between hidden and visible neurons. We modify the Restricted Boltzmann Machine to accommodate arbitrary ground states by introducing essential non-local connections efficiently.
  arXiv  Detail & Related papers  (2024-07-01T16:44:43Z)
• Transferable Boltzmann Generators [6.36887173111677]
  We propose a first framework for Boltzmann Generators that are transferable across chemical space. We show that our proposed architecture enhances the efficiency of Boltzmann Generators trained on single molecular systems.
  arXiv  Detail & Related papers  (2024-06-20T15:50:12Z)
• Monotone deep Boltzmann machines [86.50247625239406]
  Deep Boltzmann machines (DBMs) are multi-layered probabilistic models governed by a pairwise energy function. We develop a new class of restricted model, the monotone DBM, which allows for arbitrary self-connection in each layer. We show that a particular choice of activation results in a fixed-point iteration that gives a variational mean-field solution.
  arXiv  Detail & Related papers  (2023-07-11T03:02:44Z)
• Analogy between Boltzmann machines and Feynman path integrals [0.0]
  We provide a detailed exposition of the connections between Boltzmann machines and Feynman path-integral formalisms. We are able to provide general quantum circuit models that are applicable to both Boltzmann machines and to Feynman path integral descriptions.
  arXiv  Detail & Related papers  (2023-01-15T23:54:21Z)
• Gating Dropout: Communication-efficient Regularization for Sparsely Activated Transformers [78.77361169167149]
  We propose emphGating Dropout, which allows tokens to ignore the gating network and stay at their local machines. Similar to traditional dropout, we also show that Gating Dropout has a regularization effect during training, resulting in improved generalization performance.
  arXiv  Detail & Related papers  (2022-05-28T05:12:43Z)
• Oops I Took A Gradient: Scalable Sampling for Discrete Distributions [53.3142984019796]
  We show that this approach outperforms generic samplers in a number of difficult settings. We also demonstrate the use of our improved sampler for training deep energy-based models on high dimensional discrete data.
  arXiv  Detail & Related papers  (2021-02-08T20:08:50Z)
• Defence against adversarial attacks using classical and quantum-enhanced Boltzmann machines [64.62510681492994]
  generative models attempt to learn the distribution underlying a dataset, making them inherently more robust to small perturbations. We find improvements ranging from 5% to 72% against attacks with Boltzmann machines on the MNIST dataset.
  arXiv  Detail & Related papers  (2020-12-21T19:00:03Z)
• Generative Semantic Hashing Enhanced via Boltzmann Machines [61.688380278649056]
  Existing generative-hashing methods mostly assume a factorized form for the posterior distribution. We propose to employ the distribution of Boltzmann machine as the retrievalal posterior. We show that by effectively modeling correlations among different bits within a hash code, our model can achieve significant performance gains.
  arXiv  Detail & Related papers  (2020-06-16T01:23:39Z)
• Entropy, Free Energy, and Work of Restricted Boltzmann Machines [0.08594140167290096]
  We analyze the training process of the restricted Boltzmann machine in the context of statistical physics. We demonstrate the growth of the correlation between the visible and hidden layers via the subadditivity of entropies as the training proceeds. We discuss the Jarzynski equality which connects the path average of the exponential function of the work and the difference in free energies before and after training.
  arXiv  Detail & Related papers  (2020-04-10T04:16:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.