p-Adic Statistical Field Theory and Deep Belief Networks

• URL: http://arxiv.org/abs/2207.13877v1
• Date: Thu, 28 Jul 2022 04:04:48 GMT
• Title: p-Adic Statistical Field Theory and Deep Belief Networks
• Authors: W. A. Z\'u\~niga-Galindo
• Abstract summary: In general quantum field theories over a $p$-adic spacetime can be formulated in a rigorous way. We show these theories are deeply connected with the deep belief networks (DBNs) In our approach, a $p$-adic SFT corresponds to a $p$-adic continuous DBN, and a discretization of this theory corresponds to a $p$-adic discrete DBN.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this work we initiate the study of the correspondence between $p$-adic statistical field theories (SFTs) and neural networks (NNs). In general quantum field theories over a $p$-adic spacetime can be formulated in a rigorous way. Nowadays these theories are considered just mathematical toy models for understanding the problems of the true theories. In this work we show these theories are deeply connected with the deep belief networks (DBNs). Hinton et al. constructed DBNs by stacking several restricted Boltzmann machines (RBMs). The purpose of this construction is to obtain a network with a hierarchical structure (a deep learning architecture). An RBM corresponds a certain spin glass, thus a DBN should correspond to an ultrametric (hierarchical) spin glass. A model of such system can be easily constructed by using $p$-adic numbers. In our approach, a $p$-adic SFT corresponds to a $p$-adic continuous DBN, and a discretization of this theory corresponds to a $p$-adic discrete DBN. We show that these last machines are universal approximators. In the $p$-adic framework, the correspondence between SFTs and NNs is not fully developed. We point out several open problems.

Related papers

• The Quantified Boolean Bayesian Network: Theory and Experiments with a Logical Graphical Model [0.0]
  We show how a probabilistic Network can be configured to represent the logical reasoning underlying human language. For inference, we investigate the use of Loopy Belief Propagation (LBP), which is not guaranteed to converge. Our experiments show that LBP indeed does converge very reliably, and our analysis shows that a round of LBP takes time $O(N2n)$, where $N$ bounds the number of variables considered.
  arXiv  Detail & Related papers  (2024-02-09T17:15:45Z)
• Jaynes Machine: The universal microstructure of deep neural networks [0.9086679566009702]
  We predict that all highly connected layers of deep neural networks have a universal microstructure of connection strengths that is distributed lognormally ($LN(mu, sigma)$). Under ideal conditions, the theory predicts that $mu$ and $sigma$ are the same for all layers in all networks.
  arXiv  Detail & Related papers  (2023-10-10T19:22:01Z)
• On the Computational Complexity and Formal Hierarchy of Second Order Recurrent Neural Networks [59.85314067235965]
  We extend the theoretical foundation for the $2nd$-order recurrent network ($2nd$ RNN) We prove there exists a class of a $2nd$ RNN that is Turing-complete with bounded time. We also demonstrate that $2$nd order RNNs, without memory, outperform modern-day models such as vanilla RNNs and gated recurrent units in recognizing regular grammars.
  arXiv  Detail & Related papers  (2023-09-26T06:06:47Z)
• Neural Network Field Theories: Non-Gaussianity, Actions, and Locality [0.0]
  Both the path integral measure in field theory and ensembles of neural networks describe distributions over functions. An expansion in $1/N$ corresponds to interactions in the field theory, but others, such as in a small breaking of the statistical independence of network parameters, can also lead to interacting theories.
  arXiv  Detail & Related papers  (2023-07-06T18:00:01Z)
• Effective Minkowski Dimension of Deep Nonparametric Regression: Function Approximation and Statistical Theories [70.90012822736988]
  Existing theories on deep nonparametric regression have shown that when the input data lie on a low-dimensional manifold, deep neural networks can adapt to intrinsic data structures. This paper introduces a relaxed assumption that input data are concentrated around a subset of $mathbbRd$ denoted by $mathcalS$, and the intrinsic dimension $mathcalS$ can be characterized by a new complexity notation -- effective Minkowski dimension.
  arXiv  Detail & Related papers  (2023-06-26T17:13:31Z)
• Spatially heterogeneous learning by a deep student machine [0.0]
  Deep neural networks (DNN) with a huge number of adjustable parameters remain largely black boxes. We study supervised learning by a DNN of width $N$ and depth $L$ consisting of $NL$ perceptrons with $c$ inputs by a statistical mechanics approach called the teacher-student setting. We show that the problem becomes exactly solvable in what we call as 'dense limit': $N gg c gg 1$ and $M gg 1$ with fixed $alpha=M/c$ using the replica method developed in (H. Yoshino, (
  arXiv  Detail & Related papers  (2023-02-15T01:09:03Z)
• Convolutional Neural Networks on Manifolds: From Graphs and Back [122.06927400759021]
  We propose a manifold neural network (MNN) composed of a bank of manifold convolutional filters and point-wise nonlinearities. To sum up, we focus on the manifold model as the limit of large graphs and construct MNNs, while we can still bring back graph neural networks by the discretization of MNNs.
  arXiv  Detail & Related papers  (2022-10-01T21:17:39Z)
• Understanding Deep Neural Function Approximation in Reinforcement Learning via $\epsilon$-Greedy Exploration [53.90873926758026]
  This paper provides a theoretical study of deep neural function approximation in reinforcement learning (RL) We focus on the value based algorithm with the $epsilon$-greedy exploration via deep (and two-layer) neural networks endowed by Besov (and Barron) function spaces. Our analysis reformulates the temporal difference error in an $L2(mathrmdmu)$-integrable space over a certain averaged measure $mu$, and transforms it to a generalization problem under the non-iid setting.
  arXiv  Detail & Related papers  (2022-09-15T15:42:47Z)
• Unified Field Theory for Deep and Recurrent Neural Networks [56.735884560668985]
  We present a unified and systematic derivation of the mean-field theory for both recurrent and deep networks. We find that convergence towards the mean-field theory is typically slower for recurrent networks than for deep networks. Our method exposes that Gaussian processes are but the lowest order of a systematic expansion in $1/n$.
  arXiv  Detail & Related papers  (2021-12-10T15:06:11Z)
• Quantum field-theoretic machine learning [0.0]
  We recast the $phi4$ scalar field theory as a machine learning algorithm within the mathematically rigorous framework of Markov random fields. Neural networks are additionally derived from the $phi4$ theory which can be viewed as generalizations of conventional neural networks.
  arXiv  Detail & Related papers  (2021-02-18T16:12:51Z)
• A simple geometric proof for the benefit of depth in ReLU networks [57.815699322370826]
  We present a simple proof for the benefit of depth in multi-layer feedforward network with rectified activation ("depth separation") We present a concrete neural network with linear depth (in $m$) and small constant width ($leq 4$) that classifies the problem with zero error.
  arXiv  Detail & Related papers  (2021-01-18T15:40:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.