Neural tensor contractions and the expressive power of deep neural quantum states

• URL: http://arxiv.org/abs/2103.10293v1
• Date: Thu, 18 Mar 2021 14:47:38 GMT
• Title: Neural tensor contractions and the expressive power of deep neural quantum states
• Authors: Or Sharir, Amnon Shashua and Giuseppe Carleo
• Abstract summary: We establish a direct connection between general tensor networks and deep feed-forward artificial neural networks. We show that neural-network states have strictly the same or higher expressive power than practically usable variational tensor networks.
• Score: 17.181118551107453
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We establish a direct connection between general tensor networks and deep feed-forward artificial neural networks. The core of our results is the construction of neural-network layers that efficiently perform tensor contractions, and that use commonly adopted non-linear activation functions. The resulting deep networks feature a number of edges that closely matches the contraction complexity of the tensor networks to be approximated. In the context of many-body quantum states, this result establishes that neural-network states have strictly the same or higher expressive power than practically usable variational tensor networks. As an example, we show that all matrix product states can be efficiently written as neural-network states with a number of edges polynomial in the bond dimension and depth logarithmic in the system size. The opposite instead does not hold true, and our results imply that there exist quantum states that are not efficiently expressible in terms of matrix product states or practically usable PEPS, but that are instead efficiently expressible with neural network states.

Related papers

• Dense Neural Networks are not Universal Approximators [53.27010448621372]
  We show that dense neural networks do not possess universality of arbitrary continuous functions.<n>We consider ReLU neural networks subject to natural constraints on weights and input and output dimensions.
  arXiv  Detail & Related papers  (2026-02-07T16:52:38Z)
• Toward a Physics of Deep Learning and Brains [0.0]
  We show that equations used to describe neuronal avalanches in living brains can also be applied to cascades of activity in deep neural networks.<n>This theoretical framework demonstrates that universal features are shared by both biological and artificial neural networks.
  arXiv  Detail & Related papers  (2025-09-26T17:59:57Z)
• Graph Neural Networks for Learning Equivariant Representations of Neural Networks [55.04145324152541]
  We propose to represent neural networks as computational graphs of parameters. Our approach enables a single model to encode neural computational graphs with diverse architectures. We showcase the effectiveness of our method on a wide range of tasks, including classification and editing of implicit neural representations.
  arXiv  Detail & Related papers  (2024-03-18T18:01:01Z)
• Towards Efficient Verification of Quantized Neural Networks [9.352320240912109]
  Quantization replaces floating point arithmetic with integer arithmetic in deep neural network models. We show how efficiency can be improved by utilizing gradient-based search methods and also bound-propagation techniques.
  arXiv  Detail & Related papers  (2023-12-20T00:43:13Z)
• Entanglement transition in deep neural quantum states [0.0]
  We show how information propagation in deep neural networks impacts the physical entanglement properties of deep neural quantum states. With this bridge we can identify optimal neural quantum state hyper parameter regimes for representing area as well as volume law entangled states. This advance of our understanding of network configurations for accurate quantum state representation helps to develop effective representations to deal with volume-law quantum states.
  arXiv  Detail & Related papers  (2023-12-19T08:40:23Z)
• Addressing caveats of neural persistence with deep graph persistence [54.424983583720675]
  We find that the variance of network weights and spatial concentration of large weights are the main factors that impact neural persistence. We propose an extension of the filtration underlying neural persistence to the whole neural network instead of single layers. This yields our deep graph persistence measure, which implicitly incorporates persistent paths through the network and alleviates variance-related issues.
  arXiv  Detail & Related papers  (2023-07-20T13:34:11Z)
• Extrapolation and Spectral Bias of Neural Nets with Hadamard Product: a Polynomial Net Study [55.12108376616355]
  The study on NTK has been devoted to typical neural network architectures, but is incomplete for neural networks with Hadamard products (NNs-Hp) In this work, we derive the finite-width-K formulation for a special class of NNs-Hp, i.e., neural networks. We prove their equivalence to the kernel regression predictor with the associated NTK, which expands the application scope of NTK.
  arXiv  Detail & Related papers  (2022-09-16T06:36:06Z)
• Variable Bitrate Neural Fields [75.24672452527795]
  We present a dictionary method for compressing feature grids, reducing their memory consumption by up to 100x. We formulate the dictionary optimization as a vector-quantized auto-decoder problem which lets us learn end-to-end discrete neural representations in a space where no direct supervision is available.
  arXiv  Detail & Related papers  (2022-06-15T17:58:34Z)
• Tensor Network States with Low-Rank Tensors [6.385624548310884]
  We introduce the idea of imposing low-rank constraints on the tensors that compose the tensor network. With this modification, the time and complexities for the network optimization can be substantially reduced. We find that choosing the tensor rank $r$ to be on the order of the bond $m$, is sufficient to obtain high-accuracy groundstate approximations.
  arXiv  Detail & Related papers  (2022-05-30T17:58:16Z)
• Fourier Neural Networks for Function Approximation [2.840363325289377]
  It is proved extensively that neural networks are universal approximators. It is specifically proved that for a narrow neural network to approximate a function which is otherwise implemented by a deep Neural network, the network take exponentially large number of neurons.
  arXiv  Detail & Related papers  (2021-10-21T09:30:26Z)
• The Connection Between Approximation, Depth Separation and Learnability in Neural Networks [70.55686685872008]
  We study the connection between learnability and approximation capacity. We show that learnability with deep networks of a target function depends on the ability of simpler classes to approximate the target.
  arXiv  Detail & Related papers  (2021-01-31T11:32:30Z)
• Block-term Tensor Neural Networks [29.442026567710435]
  We show that block-term tensor layers (BT-layers) can be easily adapted to neural network models, such as CNNs and RNNs. BT-layers in CNNs and RNNs can achieve a very large compression ratio on the number of parameters while preserving or improving the representation power of the original DNNs.
  arXiv  Detail & Related papers  (2020-10-10T09:58:43Z)
• Reservoir Memory Machines as Neural Computers [70.5993855765376]
  Differentiable neural computers extend artificial neural networks with an explicit memory without interference. We achieve some of the computational capabilities of differentiable neural computers with a model that can be trained very efficiently.
  arXiv  Detail & Related papers  (2020-09-14T12:01:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.