Restricted Boltzmann Machine, recent advances and mean-field theory
- URL: http://arxiv.org/abs/2011.11307v2
- Date: Fri, 28 May 2021 08:32:54 GMT
- Title: Restricted Boltzmann Machine, recent advances and mean-field theory
- Authors: Aur\'elien Decelle and Cyril Furtlehner
- Abstract summary: Review deals with Restricted Boltzmann Machine (RBM) under the light of statistical physics.
RBM is a classical family of Machine learning (ML) models which played a central role in the development of deep learning.
- Score: 0.8702432681310401
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: This review deals with Restricted Boltzmann Machine (RBM) under the light of
statistical physics. The RBM is a classical family of Machine learning (ML)
models which played a central role in the development of deep learning. Viewing
it as a Spin Glass model and exhibiting various links with other models of
statistical physics, we gather recent results dealing with mean-field theory in
this context. First the functioning of the RBM can be analyzed via the phase
diagrams obtained for various statistical ensembles of RBM leading in
particular to identify a {\it compositional phase} where a small number of
features or modes are combined to form complex patterns. Then we discuss recent
works either able to devise mean-field based learning algorithms; either able
to reproduce generic aspects of the learning process from some {\it ensemble
dynamics equations} or/and from linear stability arguments.
Related papers
- Relational Learning in Pre-Trained Models: A Theory from Hypergraph Recovery Perspective [60.64922606733441]
We introduce a mathematical model that formalizes relational learning as hypergraph recovery to study pre-training of Foundation Models (FMs)
In our framework, the world is represented as a hypergraph, with data abstracted as random samples from hyperedges. We theoretically examine the feasibility of a Pre-Trained Model (PTM) to recover this hypergraph and analyze the data efficiency in a minimax near-optimal style.
arXiv Detail & Related papers (2024-06-17T06:20:39Z) - Discovering Interpretable Physical Models using Symbolic Regression and
Discrete Exterior Calculus [55.2480439325792]
We propose a framework that combines Symbolic Regression (SR) and Discrete Exterior Calculus (DEC) for the automated discovery of physical models.
DEC provides building blocks for the discrete analogue of field theories, which are beyond the state-of-the-art applications of SR to physical problems.
We prove the effectiveness of our methodology by re-discovering three models of Continuum Physics from synthetic experimental data.
arXiv Detail & Related papers (2023-10-10T13:23:05Z) - Tasks Makyth Models: Machine Learning Assisted Surrogates for Tipping
Points [0.0]
We present a machine learning (ML)-assisted framework for detecting tipping points in the emergent behavior of complex systems.
We construct reduced-order models for the emergent dynamics at different scales.
We contrast the uses of the different models and the effort involved in learning them.
arXiv Detail & Related papers (2023-09-25T17:58:23Z) - Capturing dynamical correlations using implicit neural representations [85.66456606776552]
We develop an artificial intelligence framework which combines a neural network trained to mimic simulated data from a model Hamiltonian with automatic differentiation to recover unknown parameters from experimental data.
In doing so, we illustrate the ability to build and train a differentiable model only once, which then can be applied in real-time to multi-dimensional scattering data.
arXiv Detail & Related papers (2023-04-08T07:55:36Z) - The emergence of a concept in shallow neural networks [0.0]
We consider restricted Boltzmann machine (RBMs) trained over an unstructured dataset made of blurred copies of definite but unavailable archetypes''
We show that there exists a critical sample size beyond which the RBM can learn archetypes.
arXiv Detail & Related papers (2021-09-01T15:56:38Z) - Mean-field methods and algorithmic perspectives for high-dimensional
machine learning [5.406386303264086]
We revisit an approach based on the tools of statistical physics of disordered systems.
We capitalize on the deep connection between the replica method and message passing algorithms in order to shed light on the phase diagrams of various theoretical models.
arXiv Detail & Related papers (2021-03-10T09:02:36Z) - Physics-Integrated Variational Autoencoders for Robust and Interpretable
Generative Modeling [86.9726984929758]
We focus on the integration of incomplete physics models into deep generative models.
We propose a VAE architecture in which a part of the latent space is grounded by physics.
We demonstrate generative performance improvements over a set of synthetic and real-world datasets.
arXiv Detail & Related papers (2021-02-25T20:28:52Z) - Using Restricted Boltzmann Machines to Model Molecular Geometries [0.0]
This paper proposes a new methodology for modeling a set of physical parameters by taking advantage of the Boltzmann machine's fast learning capacity and representational power.
In this paper we introduce a new RBM based on the Tanh activation function, and conduct a comparison of RBMs with different activation functions.
We demonstrate the ability of Gaussian RBMs to model small molecules such as water and ethane.
arXiv Detail & Related papers (2020-12-13T07:02:32Z) - Exact representations of many body interactions with RBM neural networks [77.34726150561087]
We exploit the representation power of RBMs to provide an exact decomposition of many-body contact interactions into one-body operators.
This construction generalizes the well known Hirsch's transform used for the Hubbard model to more complicated theories such as Pionless EFT in nuclear physics.
arXiv Detail & Related papers (2020-05-07T15:59:29Z) - Kernel and Rich Regimes in Overparametrized Models [69.40899443842443]
We show that gradient descent on overparametrized multilayer networks can induce rich implicit biases that are not RKHS norms.
We also demonstrate this transition empirically for more complex matrix factorization models and multilayer non-linear networks.
arXiv Detail & Related papers (2020-02-20T15:43:02Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.