Theory of overparametrization in quantum neural networks

• URL: http://arxiv.org/abs/2109.11676v1
• Date: Thu, 23 Sep 2021 22:39:48 GMT
• Title: Theory of overparametrization in quantum neural networks
• Authors: Martin Larocca, Nathan Ju, Diego Garc\'ia-Mart\'in, Patrick J. Coles, M. Cerezo
• Abstract summary: We rigorously analyze the overparametrization phenomenon in Quantum Neural Networks (QNNs) with periodic structure. Our results show that the dimension of the Lie algebra obtained from the generators of the QNN is an upper bound for $M_c$. We then connect the notion of overparametrization to the QNN capacity, so that when a QNN is overparametrized, its capacity achieves its maximum possible value.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The prospect of achieving quantum advantage with Quantum Neural Networks (QNNs) is exciting. Understanding how QNN properties (e.g., the number of parameters $M$) affect the loss landscape is crucial to the design of scalable QNN architectures. Here, we rigorously analyze the overparametrization phenomenon in QNNs with periodic structure. We define overparametrization as the regime where the QNN has more than a critical number of parameters $M_c$ that allows it to explore all relevant directions in state space. Our main results show that the dimension of the Lie algebra obtained from the generators of the QNN is an upper bound for $M_c$, and for the maximal rank that the quantum Fisher information and Hessian matrices can reach. Underparametrized QNNs have spurious local minima in the loss landscape that start disappearing when $M\geq M_c$. Thus, the overparametrization onset corresponds to a computational phase transition where the QNN trainability is greatly improved by a more favorable landscape. We then connect the notion of overparametrization to the QNN capacity, so that when a QNN is overparametrized, its capacity achieves its maximum possible value. We run numerical simulations for eigensolver, compilation, and autoencoding applications to showcase the overparametrization computational phase transition. We note that our results also apply to variational quantum algorithms and quantum optimal control.

Related papers

• Projected Stochastic Gradient Descent with Quantum Annealed Binary Gradients [51.82488018573326]
  We present QP-SBGD, a novel layer-wise optimiser tailored towards training neural networks with binary weights. BNNs reduce the computational requirements and energy consumption of deep learning models with minimal loss in accuracy. Our algorithm is implemented layer-wise, making it suitable to train larger networks on resource-limited quantum hardware.
  arXiv  Detail & Related papers  (2023-10-23T17:32:38Z)
• Variational Quantum Neural Networks (VQNNS) in Image Classification [0.0]
  This paper investigates how training of quantum neural network (QNNs) can be done using quantum optimization algorithms. In this paper, a QNN structure is made where a variational parameterized circuit is incorporated as an input layer named as Variational Quantum Neural Network (VQNNs) VQNNs is experimented with MNIST digit recognition (less complex) and crack image classification datasets which converge the computation in lesser time than QNN with decent training accuracy.
  arXiv  Detail & Related papers  (2023-03-10T11:24:32Z)
• Theoretical Guarantees for Permutation-Equivariant Quantum Neural Networks [0.0]
  We show how to build equivariant quantum neural networks (QNNs) We prove that they do not suffer from barren plateaus, quickly reach overparametrization, and generalize well from small amounts of data. Our work provides the first theoretical guarantees for equivariant QNNs, thus indicating the extreme power and potential of GQML.
  arXiv  Detail & Related papers  (2022-10-18T16:35:44Z)
• Symmetric Pruning in Quantum Neural Networks [111.438286016951]
  Quantum neural networks (QNNs) exert the power of modern quantum machines. QNNs with handcraft symmetric ansatzes generally experience better trainability than those with asymmetric ansatzes. We propose the effective quantum neural tangent kernel (EQNTK) to quantify the convergence of QNNs towards the global optima.
  arXiv  Detail & Related papers  (2022-08-30T08:17:55Z)
• GHN-Q: Parameter Prediction for Unseen Quantized Convolutional Architectures via Graph Hypernetworks [80.29667394618625]
  We conduct the first-ever study exploring the use of graph hypernetworks for predicting parameters of unseen quantized CNN architectures. We focus on a reduced CNN search space and find that GHN-Q can in fact predict quantization-robust parameters for various 8-bit quantized CNNs.
  arXiv  Detail & Related papers  (2022-08-26T08:00:02Z)
• Toward Trainability of Deep Quantum Neural Networks [87.04438831673063]
  Quantum Neural Networks (QNNs) with random structures have poor trainability due to the exponentially vanishing gradient as the circuit depth and the qubit number increase. We provide the first viable solution to the vanishing gradient problem for deep QNNs with theoretical guarantees.
  arXiv  Detail & Related papers  (2021-12-30T10:27:08Z)
• Toward Trainability of Quantum Neural Networks [87.04438831673063]
  Quantum Neural Networks (QNNs) have been proposed as generalizations of classical neural networks to achieve the quantum speed-up. Serious bottlenecks exist for training QNNs due to the vanishing with gradient rate exponential to the input qubit number. We show that QNNs with tree tensor and step controlled structures for the application of binary classification. Simulations show faster convergent rates and better accuracy compared to QNNs with random structures.
  arXiv  Detail & Related papers  (2020-11-12T08:32:04Z)
• Absence of Barren Plateaus in Quantum Convolutional Neural Networks [0.0]
  Quantum Convolutional Neural Networks (QCNNs) have been proposed. We rigorously analyze the gradient scaling for the parameters in the QCNN architecture.
  arXiv  Detail & Related papers  (2020-11-05T16:46:13Z)
• On the learnability of quantum neural networks [132.1981461292324]
  We consider the learnability of the quantum neural network (QNN) built on the variational hybrid quantum-classical scheme. We show that if a concept can be efficiently learned by QNN, then it can also be effectively learned by QNN even with gate noise.
  arXiv  Detail & Related papers  (2020-07-24T06:34:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.