Quantum-Enhanced Weight Optimization for Neural Networks Using Grover's Algorithm

• URL: http://arxiv.org/abs/2504.14568v1
• Date: Sun, 20 Apr 2025 10:59:04 GMT
• Title: Quantum-Enhanced Weight Optimization for Neural Networks Using Grover's Algorithm
• Authors: Stefan-Alexandru Jura, Mihai Udrescu,
• Abstract summary: We propose to use quantum computing in order to optimize the weights of a classical NN.<n>We design an instance of Grover's quantum search algorithm to accelerate the search for the optimal parameters of an NN.<n>Our method requires a much smaller number of qubits compared to other QNN approaches.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The main approach to hybrid quantum-classical neural networks (QNN) is employing quantum computing to build a neural network (NN) that has quantum features, which is then optimized classically. Here, we propose a different strategy: to use quantum computing in order to optimize the weights of a classical NN. As such, we design an instance of Grover's quantum search algorithm to accelerate the search for the optimal parameters of an NN during the training process, a task traditionally performed using the backpropagation algorithm with the gradient descent method. Indeed, gradient descent has issues such as exploding gradient, vanishing gradient, or convexity problem. Other methods tried to address such issues with strategies like genetic searches, but they carry additional problems like convergence consistency. Our original method avoids these issues -- because it does not calculate gradients -- and capitalizes on classical architectures' robustness and Grover's quadratic speedup in high-dimensional search spaces to significantly reduce test loss (58.75%) and improve test accuracy (35.25%), compared to classical NN weight optimization, on small datasets. Unlike most QNNs that are trained on small datasets only, our method is also scalable, as it allows the optimization of deep networks; for an NN with 3 hidden layers, trained on the Digits dataset from scikit-learn, we obtained a mean accuracy of 97.7%. Moreover, our method requires a much smaller number of qubits compared to other QNN approaches, making it very practical for near-future quantum computers that will still deliver a limited number of logical qubits.

Related papers

• Optimizing Quantum Convolutional Neural Network Architectures for Arbitrary Data Dimension [2.9396076967931526]
  Quantum convolutional neural networks (QCNNs) represent a promising approach in quantum machine learning. We propose a QCNN architecture capable of handling arbitrary input data dimensions while optimizing the allocation of quantum resources.
  arXiv  Detail & Related papers  (2024-03-28T02:25:12Z)
• Learning To Optimize Quantum Neural Network Without Gradients [3.9848482919377006]
  We introduce a novel meta-optimization algorithm that trains a emphmeta-optimizer network to output parameters for the quantum circuit. We show that we achieve a better quality minima in fewer circuit evaluations than existing gradient based algorithms on different datasets.
  arXiv  Detail & Related papers  (2023-04-15T01:09:12Z)
• Globally Optimal Training of Neural Networks with Threshold Activation Functions [63.03759813952481]
  We study weight decay regularized training problems of deep neural networks with threshold activations. We derive a simplified convex optimization formulation when the dataset can be shattered at a certain layer of the network.
  arXiv  Detail & Related papers  (2023-03-06T18:59:13Z)
• AskewSGD : An Annealed interval-constrained Optimisation method to train Quantized Neural Networks [12.229154524476405]
  We develop a new algorithm, Annealed Skewed SGD - AskewSGD - for training deep neural networks (DNNs) with quantized weights. Unlike algorithms with active sets and feasible directions, AskewSGD avoids projections or optimization under the entire feasible set. Experimental results show that the AskewSGD algorithm performs better than or on par with state of the art methods in classical benchmarks.
  arXiv  Detail & Related papers  (2022-11-07T18:13:44Z)
• Accelerating the training of single-layer binary neural networks using the HHL quantum algorithm [58.720142291102135]
  We show that useful information can be extracted from the quantum-mechanical implementation of Harrow-Hassidim-Lloyd (HHL) This paper shows, however, that useful information can be extracted from the quantum-mechanical implementation of HHL, and used to reduce the complexity of finding the solution on the classical side.
  arXiv  Detail & Related papers  (2022-10-23T11:58:05Z)
• Optimizing Tensor Network Contraction Using Reinforcement Learning [86.05566365115729]
  We propose a Reinforcement Learning (RL) approach combined with Graph Neural Networks (GNN) to address the contraction ordering problem. The problem is extremely challenging due to the huge search space, the heavy-tailed reward distribution, and the challenging credit assignment. We show how a carefully implemented RL-agent that uses a GNN as the basic policy construct can address these challenges.
  arXiv  Detail & Related papers  (2022-04-18T21:45:13Z)
• Training Quantized Deep Neural Networks via Cooperative Coevolution [27.967480639403796]
  We propose a new method for quantizing deep neural networks (DNNs) Under the framework of cooperative coevolution, we use the estimation of distribution algorithm to search for the low-bits weights. Experiments show that our method can train 4 bit ResNet-20 on the Cifar-10 dataset without sacrificing accuracy.
  arXiv  Detail & Related papers  (2021-12-23T09:13:13Z)
• Cluster-Promoting Quantization with Bit-Drop for Minimizing Network Quantization Loss [61.26793005355441]
  Cluster-Promoting Quantization (CPQ) finds the optimal quantization grids for neural networks. DropBits is a new bit-drop technique that revises the standard dropout regularization to randomly drop bits instead of neurons. We experimentally validate our method on various benchmark datasets and network architectures.
  arXiv  Detail & Related papers  (2021-09-05T15:15:07Z)
• A quantum algorithm for training wide and deep classical neural networks [72.2614468437919]
  We show that conditions amenable to classical trainability via gradient descent coincide with those necessary for efficiently solving quantum linear systems. We numerically demonstrate that the MNIST image dataset satisfies such conditions. We provide empirical evidence for $O(log n)$ training of a convolutional neural network with pooling.
  arXiv  Detail & Related papers  (2021-07-19T23:41:03Z)
• GradFreeBits: Gradient Free Bit Allocation for Dynamic Low Precision Neural Networks [4.511923587827301]
  Quantized neural networks (QNNs) are among the main approaches for deploying deep neural networks on low resource edge devices. We propose GradFreeBits: a novel joint optimization scheme for training dynamic QNNs. Our method achieves better or on par performance with current state of the art low precision neural networks on CIFAR10/100 and ImageNet classification.
  arXiv  Detail & Related papers  (2021-02-18T12:18:09Z)
• Exploring the Uncertainty Properties of Neural Networks' Implicit Priors in the Infinite-Width Limit [47.324627920761685]
  We use recent theoretical advances that characterize the function-space prior to an ensemble of infinitely-wide NNs as a Gaussian process. This gives us a better understanding of the implicit prior NNs place on function space. We also examine the calibration of previous approaches to classification with the NNGP.
  arXiv  Detail & Related papers  (2020-10-14T18:41:54Z)
• Searching for Low-Bit Weights in Quantized Neural Networks [129.8319019563356]
  Quantized neural networks with low-bit weights and activations are attractive for developing AI accelerators. We present to regard the discrete weights in an arbitrary quantized neural network as searchable variables, and utilize a differential method to search them accurately.
  arXiv  Detail & Related papers  (2020-09-18T09:13:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.