Boltzmann machines as two-dimensional tensor networks
- URL: http://arxiv.org/abs/2105.04130v1
- Date: Mon, 10 May 2021 06:14:49 GMT
- Title: Boltzmann machines as two-dimensional tensor networks
- Authors: Sujie Li, Feng Pan, Pengfei Zhou, Pan Zhang
- Abstract summary: We show that RBM and DBM can be exactly represented as a two-dimensional tensor network.
This representation gives an understanding of the expressive power of RBM and DBM.
Also provides an efficient tensor network contraction algorithm for the computing partition function of RBM and DBM.
- Score: 7.041258064903578
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Restricted Boltzmann machines (RBM) and deep Boltzmann machines (DBM) are
important models in machine learning, and recently found numerous applications
in quantum many-body physics. We show that there are fundamental connections
between them and tensor networks. In particular, we demonstrate that any RBM
and DBM can be exactly represented as a two-dimensional tensor network. This
representation gives an understanding of the expressive power of RBM and DBM
using entanglement structures of the tensor networks, also provides an
efficient tensor network contraction algorithm for the computing partition
function of RBM and DBM. Using numerical experiments, we demonstrate that the
proposed algorithm is much more accurate than the state-of-the-art machine
learning methods in estimating the partition function of restricted Boltzmann
machines and deep Boltzmann machines, and have potential applications in
training deep Boltzmann machines for general machine learning tasks.
Related papers
- DimOL: Dimensional Awareness as A New 'Dimension' in Operator Learning [63.5925701087252]
We introduce DimOL (Dimension-aware Operator Learning), drawing insights from dimensional analysis.
To implement DimOL, we propose the ProdLayer, which can be seamlessly integrated into FNO-based and Transformer-based PDE solvers.
Empirically, DimOL models achieve up to 48% performance gain within the PDE datasets.
arXiv Detail & Related papers (2024-10-08T10:48:50Z) - A Matrix Product State Model for Simultaneous Classification and Generation [0.8192907805418583]
Quantum machine learning (QML) is a rapidly expanding field that merges the principles of quantum computing with the techniques of machine learning.
Here, we present a novel matrix product state (MPS) model, where the MPS functions as both a classifier and a generator.
Our contributions offer insights into the mechanics of tensor network methods for generation tasks.
arXiv Detail & Related papers (2024-06-25T10:23:36Z) - Enhancing lattice kinetic schemes for fluid dynamics with Lattice-Equivariant Neural Networks [79.16635054977068]
We present a new class of equivariant neural networks, dubbed Lattice-Equivariant Neural Networks (LENNs)
Our approach develops within a recently introduced framework aimed at learning neural network-based surrogate models Lattice Boltzmann collision operators.
Our work opens towards practical utilization of machine learning-augmented Lattice Boltzmann CFD in real-world simulations.
arXiv Detail & Related papers (2024-05-22T17:23:15Z) - Mechanistic Neural Networks for Scientific Machine Learning [58.99592521721158]
We present Mechanistic Neural Networks, a neural network design for machine learning applications in the sciences.
It incorporates a new Mechanistic Block in standard architectures to explicitly learn governing differential equations as representations.
Central to our approach is a novel Relaxed Linear Programming solver (NeuRLP) inspired by a technique that reduces solving linear ODEs to solving linear programs.
arXiv Detail & Related papers (2024-02-20T15:23:24Z) - Learning Restricted Boltzmann Machines with greedy quantum search [2.98017021422101]
We extend scope to the quantum computing domain and propose corresponding quantum algorithms for this problem.
Our study demonstrates that the proposed quantum algorithms yield a speedup compared to the classical algorithms for learning the structure of these two classes of RBMs.
arXiv Detail & Related papers (2023-09-25T14:56:30Z) - Monotone deep Boltzmann machines [86.50247625239406]
Deep Boltzmann machines (DBMs) are multi-layered probabilistic models governed by a pairwise energy function.
We develop a new class of restricted model, the monotone DBM, which allows for arbitrary self-connection in each layer.
We show that a particular choice of activation results in a fixed-point iteration that gives a variational mean-field solution.
arXiv Detail & Related papers (2023-07-11T03:02:44Z) - Quasi-orthogonality and intrinsic dimensions as measures of learning and
generalisation [55.80128181112308]
We show that dimensionality and quasi-orthogonality of neural networks' feature space may jointly serve as network's performance discriminants.
Our findings suggest important relationships between the networks' final performance and properties of their randomly initialised feature spaces.
arXiv Detail & Related papers (2022-03-30T21:47:32Z) - Tensor networks for unsupervised machine learning [9.897828174118974]
We present the Autoregressive Matrix Product States (AMPS), a tensor-network-based model combining the matrix product states from quantum many-body physics and the autoregressive models from machine learning.
We show that the proposed model significantly outperforms the existing tensor-network-based models and the restricted Boltzmann machines.
arXiv Detail & Related papers (2021-06-24T12:51:00Z) - Mode-Assisted Joint Training of Deep Boltzmann Machines [10.292439652458157]
We show that the performance gains of the mode-assisted training are even more dramatic for DBMs.
DBMs jointly trained with the mode-assisted algorithm can represent the same data set with orders of magnitude lower number of parameters.
arXiv Detail & Related papers (2021-02-17T04:03:30Z) - Exact representations of many body interactions with RBM neural networks [77.34726150561087]
We exploit the representation power of RBMs to provide an exact decomposition of many-body contact interactions into one-body operators.
This construction generalizes the well known Hirsch's transform used for the Hubbard model to more complicated theories such as Pionless EFT in nuclear physics.
arXiv Detail & Related papers (2020-05-07T15:59:29Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.