Higher Order Derivatives of Quantum Neural Networks with Barren Plateaus

• URL: http://arxiv.org/abs/2008.07454v2
• Date: Mon, 7 Jun 2021 15:15:14 GMT
• Title: Higher Order Derivatives of Quantum Neural Networks with Barren Plateaus
• Authors: M. Cerezo, Patrick J. Coles
• Abstract summary: We show that the elements of the Hessian are exponentially suppressed in a Barren Plateau (BP) BPs will impact optimization strategies that go beyond (first-order) gradient descent. We prove novel, general formulas that can be used to analytically evaluate any high-order partial derivative on quantum hardware.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Quantum neural networks (QNNs) offer a powerful paradigm for programming near-term quantum computers and have the potential to speedup applications ranging from data science to chemistry to materials science. However, a possible obstacle to realizing that speedup is the Barren Plateau (BP) phenomenon, whereby the gradient vanishes exponentially in the system size $n$ for certain QNN architectures. The question of whether high-order derivative information such as the Hessian could help escape a BP was recently posed in the literature. Here we show that the elements of the Hessian are exponentially suppressed in a BP, so estimating the Hessian in this situation would require a precision that scales exponentially with $n$. Hence, Hessian-based approaches do not circumvent the exponential scaling associated with BPs. We also show the exponential suppression of higher order derivatives. Hence, BPs will impact optimization strategies that go beyond (first-order) gradient descent. In deriving our results, we prove novel, general formulas that can be used to analytically evaluate any high-order partial derivative on quantum hardware. These formulas will likely have independent interest and use for training quantum neural networks (outside of the context of BPs).

Related papers

• Temporal Spiking Neural Networks with Synaptic Delay for Graph Reasoning [91.29876772547348]
  Spiking neural networks (SNNs) are investigated as biologically inspired models of neural computation. This paper reveals that SNNs, when amalgamated with synaptic delay and temporal coding, are proficient in executing (knowledge) graph reasoning.
  arXiv  Detail & Related papers  (2024-05-27T05:53:30Z)
• A Review of Barren Plateaus in Variational Quantum Computing [0.32360458646569984]
  Variational quantum computing offers a flexible computational paradigm with applications in diverse areas. A key obstacle to realizing their potential is the Barren Plateau (BP) phenomenon. This article provides a comprehensive review of the current understanding of the BP phenomenon.
  arXiv  Detail & Related papers  (2024-05-01T18:00:10Z)
• Projected Stochastic Gradient Descent with Quantum Annealed Binary Gradients [51.82488018573326]
  We present QP-SBGD, a novel layer-wise optimiser tailored towards training neural networks with binary weights. BNNs reduce the computational requirements and energy consumption of deep learning models with minimal loss in accuracy. Our algorithm is implemented layer-wise, making it suitable to train larger networks on resource-limited quantum hardware.
  arXiv  Detail & Related papers  (2023-10-23T17:32:38Z)
• Neural Basis Functions for Accelerating Solutions to High Mach Euler Equations [63.8376359764052]
  We propose an approach to solving partial differential equations (PDEs) using a set of neural networks. We regress a set of neural networks onto a reduced order Proper Orthogonal Decomposition (POD) basis. These networks are then used in combination with a branch network that ingests the parameters of the prescribed PDE to compute a reduced order approximation to the PDE.
  arXiv  Detail & Related papers  (2022-08-02T18:27:13Z)
• A temporally and spatially local spike-based backpropagation algorithm to enable training in hardware [0.0]
  Spiking Neural Networks (SNNs) have emerged as a hardware efficient architecture for classification tasks. There have been several attempts to adopt the powerful backpropagation (BP) technique used in non-spiking artificial neural networks (ANNs)
  arXiv  Detail & Related papers  (2022-07-20T08:57:53Z)
• Learning Physics-Informed Neural Networks without Stacked Back-propagation [82.26566759276105]
  We develop a novel approach that can significantly accelerate the training of Physics-Informed Neural Networks. In particular, we parameterize the PDE solution by the Gaussian smoothed model and show that, derived from Stein's Identity, the second-order derivatives can be efficiently calculated without back-propagation. Experimental results show that our proposed method can achieve competitive error compared to standard PINN training but is two orders of magnitude faster.
  arXiv  Detail & Related papers  (2022-02-18T18:07:54Z)
• Toward Trainability of Deep Quantum Neural Networks [87.04438831673063]
  Quantum Neural Networks (QNNs) with random structures have poor trainability due to the exponentially vanishing gradient as the circuit depth and the qubit number increase. We provide the first viable solution to the vanishing gradient problem for deep QNNs with theoretical guarantees.
  arXiv  Detail & Related papers  (2021-12-30T10:27:08Z)
• Bayesian Deep Learning for Partial Differential Equation Parameter Discovery with Sparse and Noisy Data [0.0]
  We propose to use Bayesian neural networks (BNN) in order to recover the full system states from measurement data. We show that it is possible to accurately capture physics of varying complexity without overfitting. We demonstrate our approach on a handful of example applied to physics and non-linear dynamics.
  arXiv  Detail & Related papers  (2021-08-05T19:43:15Z)
• Toward Trainability of Quantum Neural Networks [87.04438831673063]
  Quantum Neural Networks (QNNs) have been proposed as generalizations of classical neural networks to achieve the quantum speed-up. Serious bottlenecks exist for training QNNs due to the vanishing with gradient rate exponential to the input qubit number. We show that QNNs with tree tensor and step controlled structures for the application of binary classification. Simulations show faster convergent rates and better accuracy compared to QNNs with random structures.
  arXiv  Detail & Related papers  (2020-11-12T08:32:04Z)
• A Neural Network Perturbation Theory Based on the Born Series [0.0]
  Taylor coefficients of deep neural networks (DNNs) still appear mainly in the light of interpretability studies. This gap motivates a general formulation of neural network (NN) Taylor expansions. We show that NNs adapt their derivatives mainly to the leading order of the target function's Taylor expansion.
  arXiv  Detail & Related papers  (2020-09-07T15:54:27Z)
• Trainability of Dissipative Perceptron-Based Quantum Neural Networks [0.8258451067861933]
  We analyze the gradient scaling (and hence the trainability) for a recently proposed architecture that we called dissipative QNNs (DQNNs) We find that DQNNs can exhibit barren plateaus, i.e., gradients that vanish exponentially in the number of qubits.
  arXiv  Detail & Related papers  (2020-05-26T00:59:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.