Theory on variational high-dimensional tensor networks
- URL: http://arxiv.org/abs/2303.17452v2
- Date: Mon, 22 May 2023 04:28:30 GMT
- Title: Theory on variational high-dimensional tensor networks
- Authors: Zidu Liu, Qi Ye, Li-Wei Yu, L.-M. Duan, and Dong-Ling Deng
- Abstract summary: We investigate the emergent statistical properties of random high-dimensional-network states and the trainability of tensoral networks.
We prove that variational high-dimensional networks suffer from barren plateaus for global loss functions.
Our results pave a way for their future theoretical studies and practical applications.
- Score: 2.0307382542339485
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Tensor network methods are powerful tools for studying quantum many-body
systems. In this paper, we investigate the emergent statistical properties of
random high-dimensional tensor-network states and the trainability of
variational tensor networks. We utilize diagrammatic methods and map our
problems to the calculations of different partition functions for
high-dimensional Ising models with special structures. To address the notorious
difficulty in cracking these models, we develop a combinatorial method based on
solving the ``puzzle of polyominoes". With this method, we are able to
rigorously study statistical properties of the high dimensional random tensor
networks. We prove: (a) the entanglement entropy approaches the maximal volume
law, except for a small probability that is bounded by an inverse polynomial of
the bond dimension; (b) the typicality occurs for the expectation value of a
local observable when the bond dimension increases. In addition, we investigate
the barren plateaus (i.e., exponentially vanishing gradients) for the
high-dimensional tensor network models. We prove that such variational models
suffer from barren plateaus for global loss functions, rendering their training
processes inefficient in general. Whereas, for local loss functions, we prove
that the gradient is independent of the system size (thus no barren plateau
occurs), but decays exponentially with the distance between the region where
the local observable acts and the site that hosts the derivative parameter. Our
results uncover in a rigorous fashion some fundamental properties for
variational high-dimensional tensor networks, which paves a way for their
future theoretical studies and practical applications.
Related papers
- Quantized Fourier and Polynomial Features for more Expressive Tensor
Network Models [9.18287948559108]
We exploit the tensor structure present in the features by constraining the model weights to be an underparametrized tensor network.
We show that, for the same number of model parameters, the resulting quantized models have a higher bound on the VC-dimension as opposed to their non-quantized counterparts.
arXiv Detail & Related papers (2023-09-11T13:18:19Z) - Absence of barren plateaus and scaling of gradients in the energy optimization of isometric tensor network states [0.0]
We consider energy problems for quantum many-body systems with extensive Hamiltonians and finite-range interactions.
We prove that variational optimization problems for matrix product states, tree tensor networks, and the multiscale entanglement renormalization ansatz are free of barren plateaus.
arXiv Detail & Related papers (2023-03-31T22:49:49Z) - Low-Rank Tensor Function Representation for Multi-Dimensional Data
Recovery [52.21846313876592]
Low-rank tensor function representation (LRTFR) can continuously represent data beyond meshgrid with infinite resolution.
We develop two fundamental concepts for tensor functions, i.e., the tensor function rank and low-rank tensor function factorization.
Our method substantiates the superiority and versatility of our method as compared with state-of-the-art methods.
arXiv Detail & Related papers (2022-12-01T04:00:38Z) - Error Analysis of Tensor-Train Cross Approximation [88.83467216606778]
We provide accuracy guarantees in terms of the entire tensor for both exact and noisy measurements.
Results are verified by numerical experiments, and may have important implications for the usefulness of cross approximations for high-order tensors.
arXiv Detail & Related papers (2022-07-09T19:33:59Z) - Decimation technique for open quantum systems: a case study with
driven-dissipative bosonic chains [62.997667081978825]
Unavoidable coupling of quantum systems to external degrees of freedom leads to dissipative (non-unitary) dynamics.
We introduce a method to deal with these systems based on the calculation of (dissipative) lattice Green's function.
We illustrate the power of this method with several examples of driven-dissipative bosonic chains of increasing complexity.
arXiv Detail & Related papers (2022-02-15T19:00:09Z) - Equivariant vector field network for many-body system modeling [65.22203086172019]
Equivariant Vector Field Network (EVFN) is built on a novel equivariant basis and the associated scalarization and vectorization layers.
We evaluate our method on predicting trajectories of simulated Newton mechanics systems with both full and partially observed data.
arXiv Detail & Related papers (2021-10-26T14:26:25Z) - The Presence and Absence of Barren Plateaus in Tensor-network Based
Machine Learning [0.0]
We study the trainability of tensor-network based machine learning models by exploring the landscapes of different loss functions.
We rigorously prove that barren plateaus prevail in the training process of the machine learning algorithms with global loss functions.
arXiv Detail & Related papers (2021-08-18T18:00:03Z) - Tensor-Train Networks for Learning Predictive Modeling of
Multidimensional Data [0.0]
A promising strategy is based on tensor networks, which have been very successful in physical and chemical applications.
We show that the weights of a multidimensional regression model can be learned by means of tensor networks with the aim of performing a powerful compact representation.
An algorithm based on alternating least squares has been proposed for approximating the weights in TT-format with a reduction of computational power.
arXiv Detail & Related papers (2021-01-22T16:14:38Z) - Multipole Graph Neural Operator for Parametric Partial Differential
Equations [57.90284928158383]
One of the main challenges in using deep learning-based methods for simulating physical systems is formulating physics-based data.
We propose a novel multi-level graph neural network framework that captures interaction at all ranges with only linear complexity.
Experiments confirm our multi-graph network learns discretization-invariant solution operators to PDEs and can be evaluated in linear time.
arXiv Detail & Related papers (2020-06-16T21:56:22Z) - Anomaly Detection with Tensor Networks [2.3895981099137535]
We exploit the memory and computational efficiency of tensor networks to learn a linear transformation over a space with a dimension exponential in the number of original features.
We produce competitive results on image datasets, despite not exploiting the locality of images.
arXiv Detail & Related papers (2020-06-03T20:41:30Z) - Semiparametric Nonlinear Bipartite Graph Representation Learning with
Provable Guarantees [106.91654068632882]
We consider the bipartite graph and formalize its representation learning problem as a statistical estimation problem of parameters in a semiparametric exponential family distribution.
We show that the proposed objective is strongly convex in a neighborhood around the ground truth, so that a gradient descent-based method achieves linear convergence rate.
Our estimator is robust to any model misspecification within the exponential family, which is validated in extensive experiments.
arXiv Detail & Related papers (2020-03-02T16:40:36Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.