Critical Points in Quantum Generative Models

• URL: http://arxiv.org/abs/2109.06957v3
• Date: Thu, 12 Jan 2023 15:40:49 GMT
• Title: Critical Points in Quantum Generative Models
• Authors: Eric R. Anschuetz
• Abstract summary: We study the clustering of local minima of the loss function near the global minimum. We give the first proof of this transition in trainability, specializing to this latter class of quantum generative model.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: One of the most important properties of neural networks is the clustering of local minima of the loss function near the global minimum, enabling efficient training. Though generative models implemented on quantum computers are known to be more expressive than their traditional counterparts, it has empirically been observed that these models experience a transition in the quality of their local minima. Namely, below some critical number of parameters, all local minima are far from the global minimum in function value; above this critical parameter count, all local minima are good approximators of the global minimum. Furthermore, for a certain class of quantum generative models, this transition has empirically been observed to occur at parameter counts exponentially large in the problem size, meaning practical training of these models is out of reach. Here, we give the first proof of this transition in trainability, specializing to this latter class of quantum generative model. We use techniques inspired by those used to study the loss landscapes of classical neural networks. We also verify that our analytic results hold experimentally even at modest model sizes.

Related papers

• Exploring Channel Distinguishability in Local Neighborhoods of the Model Space in Quantum Neural Networks [0.5277756703318045]
  Quantum Neural Networks (QNNs) have emerged and gained significant attention. QNNs have been shown to be notoriously difficult to train, which we hypothesize is partially due to the architectures, called ansatzes.
  arXiv  Detail & Related papers  (2024-10-12T10:20:26Z)
• Just How Flexible are Neural Networks in Practice? [89.80474583606242]
  It is widely believed that a neural network can fit a training set containing at least as many samples as it has parameters. In practice, however, we only find solutions via our training procedure, including the gradient and regularizers, limiting flexibility.
  arXiv  Detail & Related papers  (2024-06-17T12:24:45Z)
• Learning a Sparse Neural Network using IHT [1.124958340749622]
  This paper relies on results from the domain of advanced sparse optimization, particularly those addressing nonlinear differentiable functions. As computational power for training NNs increases, so does the complexity of the models in terms of a higher number of parameters. This paper aims to investigate whether the theoretical prerequisites for such convergence are applicable in the realm of neural network (NN) training.
  arXiv  Detail & Related papers  (2024-04-29T04:10:22Z)
• Identifying overparameterization in Quantum Circuit Born Machines [1.7259898169307613]
  We study the onset of over parameterization transitions for quantum circuit Born machines, generative models that are trained using non-adversarial gradient methods. Our results indicate that fully understanding the trainability of these models remains an open question.
  arXiv  Detail & Related papers  (2023-07-06T21:05:22Z)
• Neural network enhanced measurement efficiency for molecular groundstates [63.36515347329037]
  We adapt common neural network models to learn complex groundstate wavefunctions for several molecular qubit Hamiltonians. We find that using a neural network model provides a robust improvement over using single-copy measurement outcomes alone to reconstruct observables.
  arXiv  Detail & Related papers  (2022-06-30T17:45:05Z)
• PLATON: Pruning Large Transformer Models with Upper Confidence Bound of Weight Importance [114.1541203743303]
  We propose PLATON, which captures the uncertainty of importance scores by upper confidence bound (UCB) of importance estimation. We conduct extensive experiments with several Transformer-based models on natural language understanding, question answering and image classification.
  arXiv  Detail & Related papers  (2022-06-25T05:38:39Z)
• Hyperparameter Importance of Quantum Neural Networks Across Small Datasets [1.1470070927586014]
  A quantum neural network can play a similar role to a neural network. Very little is known about suitable circuit architectures for machine learning. This work introduces new methodologies to study quantum machine learning models.
  arXiv  Detail & Related papers  (2022-06-20T20:26:20Z)
• Generalization Metrics for Practical Quantum Advantage in Generative Models [68.8204255655161]
  Generative modeling is a widely accepted natural use case for quantum computers. We construct a simple and unambiguous approach to probe practical quantum advantage for generative modeling by measuring the algorithm's generalization performance. Our simulation results show that our quantum-inspired models have up to a $68 times$ enhancement in generating unseen unique and valid samples.
  arXiv  Detail & Related papers  (2022-01-21T16:35:35Z)
• Exponentially Many Local Minima in Quantum Neural Networks [9.442139459221785]
  Quantum Neural Networks (QNNs) are important quantum applications because of their similar promises as classical neural networks. We conduct a quantitative investigation on the landscape of loss functions of QNNs and identify a class of simple yet extremely hard QNN instances for training. We empirically confirm that our constructions can indeed be hard instances in practice with typical gradient-based circuits.
  arXiv  Detail & Related papers  (2021-10-06T03:23:44Z)
• The dilemma of quantum neural networks [63.82713636522488]
  We show that quantum neural networks (QNNs) fail to provide any benefit over classical learning models. QNNs suffer from the severely limited effective model capacity, which incurs poor generalization on real-world datasets. These results force us to rethink the role of current QNNs and to design novel protocols for solving real-world problems with quantum advantages.
  arXiv  Detail & Related papers  (2021-06-09T10:41:47Z)
• Modeling from Features: a Mean-field Framework for Over-parameterized Deep Neural Networks [54.27962244835622]
  This paper proposes a new mean-field framework for over- parameterized deep neural networks (DNNs) In this framework, a DNN is represented by probability measures and functions over its features in the continuous limit. We illustrate the framework via the standard DNN and the Residual Network (Res-Net) architectures.
  arXiv  Detail & Related papers  (2020-07-03T01:37:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.