A natural NISQ model of quantum self-attention mechanism
- URL: http://arxiv.org/abs/2305.15680v1
- Date: Thu, 25 May 2023 03:09:17 GMT
- Title: A natural NISQ model of quantum self-attention mechanism
- Authors: Shangshang Shi, Zhimin Wang, Jiaxin Li, Yanan Li, Ruimin Shang,
Haiyong Zheng, Guoqiang Zhong, Yongjian Gu
- Abstract summary: Self-attention mechanism (SAM) has demonstrated remarkable success in various applications.
Quantum neural networks (QNNs) have been developed as a novel learning model.
We propose a completely natural way of implementing SAM in QNNs, resulting in the quantum self-attention mechanism (QSAM)
- Score: 11.613292674155685
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The self-attention mechanism (SAM) has demonstrated remarkable success in
various applications. However, training SAM on classical computers becomes
computationally challenging as the number of trainable parameters grows.
Quantum neural networks (QNNs) have been developed as a novel learning model
that promises to provide speedup for pattern recognition using near-term Noisy
Intermediate-Scale Quantum (NISQ) computers. In this work, we propose a
completely natural way of implementing SAM in QNNs, resulting in the quantum
self-attention mechanism (QSAM). The fundamental operations of SAM, such as
calculating attention scores and producing attention features, are realized by
only designing the data encoding and ansatz architecture appropriately. As
these are the fundamental components of QNNs, our QSAM can be executed
efficiently on near-term NISQ devices. Our QSAM models achieve better
performance in terms of both accuracy and circuit complexity on the text
categorization task. Moreover, the robustness of QSAM against various typical
types of quantum noise is demonstrated, indicating the model's suitability for
NISQ devices. The present QSAM will serve as the fundamental building blocks
for developing large models of quantum attention neural networks for quantum
advantageous applications.
Related papers
- Quantum Mixed-State Self-Attention Network [3.1280831148667105]
This paper introduces a novel Quantum Mixed-State Attention Network (QMSAN), which integrates the principles of quantum computing with classical machine learning algorithms.
QMSAN model employs a quantum attention mechanism based on mixed states, enabling efficient direct estimation of similarity between queries and keys within the quantum domain.
Our study investigates the model's robustness in different quantum noise environments, showing that QMSAN possesses commendable robustness to low noise.
arXiv Detail & Related papers (2024-03-05T11:29:05Z) - QKSAN: A Quantum Kernel Self-Attention Network [53.96779043113156]
A Quantum Kernel Self-Attention Mechanism (QKSAM) is introduced to combine the data representation merit of Quantum Kernel Methods (QKM) with the efficient information extraction capability of SAM.
A Quantum Kernel Self-Attention Network (QKSAN) framework is proposed based on QKSAM, which ingeniously incorporates the Deferred Measurement Principle (DMP) and conditional measurement techniques.
Four QKSAN sub-models are deployed on PennyLane and IBM Qiskit platforms to perform binary classification on MNIST and Fashion MNIST.
arXiv Detail & Related papers (2023-08-25T15:08:19Z) - QSAN: A Near-term Achievable Quantum Self-Attention Network [73.15524926159702]
Self-Attention Mechanism (SAM) is good at capturing the internal connections of features.
A novel Quantum Self-Attention Network (QSAN) is proposed for image classification tasks on near-term quantum devices.
arXiv Detail & Related papers (2022-07-14T12:22:51Z) - Entanglement entropy production in Quantum Neural Networks [0.0]
Quantum Neural Networks (QNN) are considered a candidate for achieving quantum advantage in the Noisy Intermediate Quantum computer (NISQ) era.
We show a universal behavior for the rate at which entanglement is created in any given QNN architecture.
We introduce new measure to characterize the entanglement production in QNNs: the entangling speed.
arXiv Detail & Related papers (2022-06-06T10:17:17Z) - Quantum Self-Attention Neural Networks for Text Classification [8.975913540662441]
We propose a new simple network architecture, called the quantum self-attention neural network (QSANN)
We introduce the self-attention mechanism into quantum neural networks and then utilize a Gaussian projected quantum self-attention serving as a sensible quantum version of self-attention.
Our method exhibits robustness to low-level quantum noises and showcases resilience to quantum neural network architectures.
arXiv Detail & Related papers (2022-05-11T16:50:46Z) - Quantum Federated Learning with Quantum Data [87.49715898878858]
Quantum machine learning (QML) has emerged as a promising field that leans on the developments in quantum computing to explore large complex machine learning problems.
This paper proposes the first fully quantum federated learning framework that can operate over quantum data and, thus, share the learning of quantum circuit parameters in a decentralized manner.
arXiv Detail & Related papers (2021-05-30T12:19:27Z) - Toward Trainability of Quantum Neural Networks [87.04438831673063]
Quantum Neural Networks (QNNs) have been proposed as generalizations of classical neural networks to achieve the quantum speed-up.
Serious bottlenecks exist for training QNNs due to the vanishing with gradient rate exponential to the input qubit number.
We show that QNNs with tree tensor and step controlled structures for the application of binary classification. Simulations show faster convergent rates and better accuracy compared to QNNs with random structures.
arXiv Detail & Related papers (2020-11-12T08:32:04Z) - On the learnability of quantum neural networks [132.1981461292324]
We consider the learnability of the quantum neural network (QNN) built on the variational hybrid quantum-classical scheme.
We show that if a concept can be efficiently learned by QNN, then it can also be effectively learned by QNN even with gate noise.
arXiv Detail & Related papers (2020-07-24T06:34:34Z) - Entanglement Classification via Neural Network Quantum States [58.720142291102135]
In this paper we combine machine-learning tools and the theory of quantum entanglement to perform entanglement classification for multipartite qubit systems in pure states.
We use a parameterisation of quantum systems using artificial neural networks in a restricted Boltzmann machine (RBM) architecture, known as Neural Network Quantum States (NNS)
arXiv Detail & Related papers (2019-12-31T07:40:23Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.