Neural Network Field Theories: Non-Gaussianity, Actions, and Locality

• URL: http://arxiv.org/abs/2307.03223v2
• Date: Wed, 13 Dec 2023 22:10:45 GMT
• Title: Neural Network Field Theories: Non-Gaussianity, Actions, and Locality
• Authors: Mehmet Demirtas, James Halverson, Anindita Maiti, Matthew D. Schwartz, Keegan Stoner
• Abstract summary: Both the path integral measure in field theory and ensembles of neural networks describe distributions over functions. An expansion in $1/N$ corresponds to interactions in the field theory, but others, such as in a small breaking of the statistical independence of network parameters, can also lead to interacting theories.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Both the path integral measure in field theory and ensembles of neural networks describe distributions over functions. When the central limit theorem can be applied in the infinite-width (infinite-$N$) limit, the ensemble of networks corresponds to a free field theory. Although an expansion in $1/N$ corresponds to interactions in the field theory, others, such as in a small breaking of the statistical independence of network parameters, can also lead to interacting theories. These other expansions can be advantageous over the $1/N$-expansion, for example by improved behavior with respect to the universal approximation theorem. Given the connected correlators of a field theory, one can systematically reconstruct the action order-by-order in the expansion parameter, using a new Feynman diagram prescription whose vertices are the connected correlators. This method is motivated by the Edgeworth expansion and allows one to derive actions for neural network field theories. Conversely, the correspondence allows one to engineer architectures realizing a given field theory by representing action deformations as deformations of neural network parameter densities. As an example, $\phi^4$ theory is realized as an infinite-$N$ neural network field theory.

Related papers

• Numerical Approximation Capacity of Neural Networks with Bounded Parameters: Do Limits Exist, and How Can They Be Measured? [4.878983382452911]
  We show that while universal approximation is theoretically feasible, in practical numerical scenarios, Deep Neural Networks (DNNs) can only be approximated by a finite-dimensional vector space. We introduce the concepts of textit$epsilon$ outer measure and textitNumerical Span Dimension (NSdim) to quantify the approximation capacity limit of a family of networks.
  arXiv  Detail & Related papers  (2024-09-25T07:43:48Z)
• Unified Field Theory for Deep and Recurrent Neural Networks [56.735884560668985]
  We present a unified and systematic derivation of the mean-field theory for both recurrent and deep networks. We find that convergence towards the mean-field theory is typically slower for recurrent networks than for deep networks. Our method exposes that Gaussian processes are but the lowest order of a systematic expansion in $1/n$.
  arXiv  Detail & Related papers  (2021-12-10T15:06:11Z)
• Building Quantum Field Theories Out of Neurons [0.0]
  An approach to field theory is studied in which fields are comprised of $N$ constituent random neurons. Gaussianity is exhibited at large-$N$, potentially explaining a feature of field theories in Nature.
  arXiv  Detail & Related papers  (2021-12-08T19:05:36Z)
• Nonperturbative renormalization for the neural network-QFT correspondence [0.0]
  We study the concepts of locality and power-counting in this context. We provide an analysis in terms of the nonperturbative renormalization group using the Wetterich-Morris equation. Our aim is to provide a useful formalism to investigate neural networks behavior beyond the large-width limit.
  arXiv  Detail & Related papers  (2021-08-03T10:36:04Z)
• The Separation Capacity of Random Neural Networks [78.25060223808936]
  We show that a sufficiently large two-layer ReLU-network with standard Gaussian weights and uniformly distributed biases can solve this problem with high probability. We quantify the relevant structure of the data in terms of a novel notion of mutual complexity.
  arXiv  Detail & Related papers  (2021-07-31T10:25:26Z)
• Quantum field-theoretic machine learning [0.0]
  We recast the $phi4$ scalar field theory as a machine learning algorithm within the mathematically rigorous framework of Markov random fields. Neural networks are additionally derived from the $phi4$ theory which can be viewed as generalizations of conventional neural networks.
  arXiv  Detail & Related papers  (2021-02-18T16:12:51Z)
• A Convergence Theory Towards Practical Over-parameterized Deep Neural Networks [56.084798078072396]
  We take a step towards closing the gap between theory and practice by significantly improving the known theoretical bounds on both the network width and the convergence time. We show that convergence to a global minimum is guaranteed for networks with quadratic widths in the sample size and linear in their depth at a time logarithmic in both. Our analysis and convergence bounds are derived via the construction of a surrogate network with fixed activation patterns that can be transformed at any time to an equivalent ReLU network of a reasonable size.
  arXiv  Detail & Related papers  (2021-01-12T00:40:45Z)
• Topological Quantum Gravity of the Ricci Flow [62.997667081978825]
  We present a family of topological quantum gravity theories associated with the geometric theory of the Ricci flow. First, we use BRST quantization to construct a "primitive" topological Lifshitz-type theory for only the spatial metric. We extend the primitive theory by gauging foliation-preserving spacetime symmetries.
  arXiv  Detail & Related papers  (2020-10-29T06:15:30Z)
• Neural Networks and Quantum Field Theory [0.0]
  We propose a theoretical understanding of neural networks in terms of Wilsonian effective field theory. The correspondence relies on the fact that many neural networks are drawn from Gaussian processes.
  arXiv  Detail & Related papers  (2020-08-19T18:00:06Z)
• Generalization bound of globally optimal non-convex neural network training: Transportation map estimation by infinite dimensional Langevin dynamics [50.83356836818667]
  We introduce a new theoretical framework to analyze deep learning optimization with connection to its generalization error. Existing frameworks such as mean field theory and neural tangent kernel theory for neural network optimization analysis typically require taking limit of infinite width of the network to show its global convergence.
  arXiv  Detail & Related papers  (2020-07-11T18:19:50Z)
• Neural Operator: Graph Kernel Network for Partial Differential Equations [57.90284928158383]
  This work is to generalize neural networks so that they can learn mappings between infinite-dimensional spaces (operators) We formulate approximation of the infinite-dimensional mapping by composing nonlinear activation functions and a class of integral operators. Experiments confirm that the proposed graph kernel network does have the desired properties and show competitive performance compared to the state of the art solvers.
  arXiv  Detail & Related papers  (2020-03-07T01:56:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.