Identifying overparameterization in Quantum Circuit Born Machines

• URL: http://arxiv.org/abs/2307.03292v2
• Date: Mon, 10 Jul 2023 11:38:15 GMT
• Title: Identifying overparameterization in Quantum Circuit Born Machines
• Authors: Andrea Delgado, Francisco Rios, Kathleen E. Hamilton
• Abstract summary: We study the onset of over parameterization transitions for quantum circuit Born machines, generative models that are trained using non-adversarial gradient methods. Our results indicate that fully understanding the trainability of these models remains an open question.
• Score: 1.7259898169307613
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In machine learning, overparameterization is associated with qualitative changes in the empirical risk landscape, which can lead to more efficient training dynamics. For many parameterized models used in statistical learning, there exists a critical number of parameters, or model size, above which the model is constructed and trained in the overparameterized regime. There are many characteristics of overparameterized loss landscapes. The most significant is the convergence of standard gradient descent to global or local minima of low loss. In this work, we study the onset of overparameterization transitions for quantum circuit Born machines, generative models that are trained using non-adversarial gradient-based methods. We observe that bounds based on numerical analysis are in general good lower bounds on the overparameterization transition. However, bounds based on the quantum circuit's algebraic structure are very loose upper bounds. Our results indicate that fully understanding the trainability of these models remains an open question.

Related papers

• Parameter Estimation in Quantum Metrology Technique for Time Series Prediction [0.0]
  The paper investigates the techniques of quantum computation in metrological predictions. It focuses on enhancing prediction potential through variational parameter estimation. The impacts of various parameter distributions and learning rates on predictive accuracy are investigated.
  arXiv  Detail & Related papers  (2024-06-12T05:55:45Z)
• Neural network analysis of neutron and X-ray reflectivity data: Incorporating prior knowledge for tackling the phase problem [141.5628276096321]
  We present an approach that utilizes prior knowledge to regularize the training process over larger parameter spaces. We demonstrate the effectiveness of our method in various scenarios, including multilayer structures with box model parameterization. In contrast to previous methods, our approach scales favorably when increasing the complexity of the inverse problem.
  arXiv  Detail & Related papers  (2023-06-28T11:15:53Z)
• Backpropagation scaling in parameterised quantum circuits [0.0]
  We introduce circuits that are not known to be classically simulable and admit gradient estimation with significantly fewer circuits. Specifically, these circuits allow for fast estimation of the gradient, higher order partial derivatives and the Fisher information matrix. In a toy classification problem on 16 qubits, such circuits show competitive performance with other methods, while reducing the training cost by about two orders of magnitude.
  arXiv  Detail & Related papers  (2023-06-26T18:00:09Z)
• Trainability Enhancement of Parameterized Quantum Circuits via Reduced-Domain Parameter Initialization [3.031137751464259]
  We show that by reducing the initial domain of each parameter proportional to the square root of circuit depth, the magnitude of the cost gradient decays at most inversely to qubit count and circuit depth. This strategy can protect specific quantum neural networks from exponentially many spurious local minima.
  arXiv  Detail & Related papers  (2023-02-14T06:41:37Z)
• Instance-Dependent Generalization Bounds via Optimal Transport [51.71650746285469]
  Existing generalization bounds fail to explain crucial factors that drive the generalization of modern neural networks. We derive instance-dependent generalization bounds that depend on the local Lipschitz regularity of the learned prediction function in the data space. We empirically analyze our generalization bounds for neural networks, showing that the bound values are meaningful and capture the effect of popular regularization methods during training.
  arXiv  Detail & Related papers  (2022-11-02T16:39:42Z)
• Quantum Lazy Training [2.492300648514128]
  We show that the training of geometrically local parameterized quantum circuits enters the lazy regime for large numbers of qubits. More precisely, we prove bounds on the rate of changes of the parameters of such a geometrically local parameterized quantum circuit in the training process.
  arXiv  Detail & Related papers  (2022-02-16T18:24:37Z)
• Mode connectivity in the loss landscape of parameterized quantum circuits [1.7546369508217283]
  Variational training of parameterized quantum circuits (PQCs) underpins many employed on near-term noisy intermediate scale quantum (NISQ) devices. We adapt the qualitative loss landscape characterization for neural networks introduced in citegoodfellowqualitatively,li 2017visualizing and tests for connectivity used in citedraxler 2018essentially to study the loss landscape features in PQC training.
  arXiv  Detail & Related papers  (2021-11-09T18:28:46Z)
• FLIP: A flexible initializer for arbitrarily-sized parametrized quantum circuits [105.54048699217668]
  We propose a FLexible Initializer for arbitrarily-sized Parametrized quantum circuits. FLIP can be applied to any family of PQCs, and instead of relying on a generic set of initial parameters, it is tailored to learn the structure of successful parameters. We illustrate the advantage of using FLIP in three scenarios: a family of problems with proven barren plateaus, PQC training to solve max-cut problem instances, and PQC training for finding the ground state energies of 1D Fermi-Hubbard models.
  arXiv  Detail & Related papers  (2021-03-15T17:38:33Z)
• On the Sparsity of Neural Machine Translation Models [65.49762428553345]
  We investigate whether redundant parameters can be reused to achieve better performance. Experiments and analyses are systematically conducted on different datasets and NMT architectures.
  arXiv  Detail & Related papers  (2020-10-06T11:47:20Z)
• Characterizing the loss landscape of variational quantum circuits [77.34726150561087]
  We introduce a way to compute the Hessian of the loss function of VQCs. We show how this information can be interpreted and compared to classical neural networks.
  arXiv  Detail & Related papers  (2020-08-06T17:48:12Z)
• Multiplicative noise and heavy tails in stochastic optimization [62.993432503309485]
  empirical optimization is central to modern machine learning, but its role in its success is still unclear. We show that it commonly arises in parameters of discrete multiplicative noise due to variance. A detailed analysis is conducted in which we describe on key factors, including recent step size, and data, all exhibit similar results on state-of-the-art neural network models.
  arXiv  Detail & Related papers  (2020-06-11T09:58:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.