A Survey of Quantum Transformers: Approaches, Advantages, Challenges, and Future Directions

• URL: http://arxiv.org/abs/2504.03192v2
• Date: Fri, 11 Apr 2025 14:00:33 GMT
• Title: A Survey of Quantum Transformers: Approaches, Advantages, Challenges, and Future Directions
• Authors: Hui Zhang, Qinglin Zhao,
• Abstract summary: Quantum Transformer models represent a significant research direction in quantum machine learning (QML)<n>PQC-based Transformer models are the primary focus of current research.<n>Quantum Linear Algebra (QLA)-based Transformer models rely on future fault-tolerant quantum computing.
• Score: 2.5871385953824855
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Quantum Transformer models represent a significant research direction in quantum machine learning (QML), leveraging the parallelism and entanglement properties of quantum computing to overcome the computational complexity and expressive limitations of classical Transformers. Parameterized quantum circuit (PQC)-based Transformer models are the primary focus of current research, employing PQCs to achieve varying degrees of quantumization, including strategies such as QKV-only Quantum mapping, Quantum Pairwise Attention, Quantum Global Attention, and Quantum-Assisted Acceleration. These approaches are well-suited to Noisy Intermediate-Scale Quantum (NISQ) devices, demonstrating potential in small-scale tasks to reduce complexity or enhance performance. The strength of PQC-based methods lies in their compatibility with existing quantum hardware, positioning them as the main pathway toward the practical implementation of quantum Transformers. However, these methods face challenges such as limited scalability, the absence of standardized testing benchmarks, and the "barren plateau" problem during training. As a complementary approach, Quantum Linear Algebra (QLA)-based Transformer models rely on future fault-tolerant quantum computing, utilizing techniques like block-encoding and Quantum Singular Value Transformation (QSVT) to achieve efficient matrix operations and theoretically significant complexity reductions, though they remain in the theoretical exploration stage. Future research should prioritize optimizing PQC-based hybrid architectures and quantum global attention models, establishing unified evaluation frameworks, and addressing training difficulties, while also exploring hybrid PQC-QLA approaches to advance the development of quantum Transformers.

Related papers

• Enhancing variational quantum algorithms by balancing training on classical and quantum hardware [1.8377902806196762]
  Variational quantum algorithms (VQAs) have the potential to provide a near-term route to quantum utility or advantage.<n>VQAs have been proposed for a multitude of tasks such as ground-state estimation.<n>There remain major challenges in its trainability and resource costs on quantum hardware.
  arXiv  Detail & Related papers  (2025-03-20T17:17:58Z)
• Leveraging Pre-Trained Neural Networks to Enhance Machine Learning with Variational Quantum Circuits [48.33631905972908]
  We introduce an innovative approach that utilizes pre-trained neural networks to enhance Variational Quantum Circuits (VQC) This technique effectively separates approximation error from qubit count and removes the need for restrictive conditions. Our results extend to applications such as human genome analysis, demonstrating the broad applicability of our approach.
  arXiv  Detail & Related papers  (2024-11-13T12:03:39Z)
• Transformer Models for Quantum Gate Set Tomography [1.1528488253382057]
  Quantum computation represents a promising frontier in the domain of high-performance computing. This study investigates the challenges of manufacturing high-fidelity and scalable quantum processors.
  arXiv  Detail & Related papers  (2024-05-03T13:45:27Z)
• Parameterized quantum comb and simpler circuits for reversing unknown qubit-unitary operations [8.14510296131348]
  We propose PQComb to harness the full potential of quantum combs for diverse quantum process transformation tasks.<n>We present two streamlined protocols for the time-reversal simulation of unknown qubit unitary evolutions.<n>We also extend PQComb to solve the problems of qutrit unitary transformation and channel discrimination.
  arXiv  Detail & Related papers  (2024-03-06T14:53:24Z)
• QuantumSEA: In-Time Sparse Exploration for Noise Adaptive Quantum Circuits [82.50620782471485]
  QuantumSEA is an in-time sparse exploration for noise-adaptive quantum circuits. It aims to achieve two key objectives: (1) implicit circuits capacity during training and (2) noise robustness. Our method establishes state-of-the-art results with only half the number of quantum gates and 2x time saving of circuit executions.
  arXiv  Detail & Related papers  (2024-01-10T22:33:00Z)
• Enabling Non-Linear Quantum Operations through Variational Quantum Splines [1.3874486202578669]
  We propose a novel method for approximating non-linear quantum activation functions using hybrid quantum-classical computation. The proposed method relies on a flexible problem representation for non-linear approximation and it is suitable to be embedded in existing quantum neural network architectures.
  arXiv  Detail & Related papers  (2023-03-08T18:39:43Z)
• Optimal Stochastic Resource Allocation for Distributed Quantum Computing [50.809738453571015]
  We propose a resource allocation scheme for distributed quantum computing (DQC) based on programming to minimize the total deployment cost for quantum resources. The evaluation demonstrates the effectiveness and ability of the proposed scheme to balance the utilization of quantum computers and on-demand quantum computers.
  arXiv  Detail & Related papers  (2022-09-16T02:37:32Z)
• Synergy Between Quantum Circuits and Tensor Networks: Short-cutting the Race to Practical Quantum Advantage [43.3054117987806]
  We introduce a scalable procedure for harnessing classical computing resources to provide pre-optimized initializations for quantum circuits. We show this method significantly improves the trainability and performance of PQCs on a variety of problems. By demonstrating a means of boosting limited quantum resources using classical computers, our approach illustrates the promise of this synergy between quantum and quantum-inspired models in quantum computing.
  arXiv  Detail & Related papers  (2022-08-29T15:24:03Z)
• Circuit Symmetry Verification Mitigates Quantum-Domain Impairments [69.33243249411113]
  We propose circuit-oriented symmetry verification that are capable of verifying the commutativity of quantum circuits without the knowledge of the quantum state. In particular, we propose the Fourier-temporal stabilizer (STS) technique, which generalizes the conventional quantum-domain formalism to circuit-oriented stabilizers.
  arXiv  Detail & Related papers  (2021-12-27T21:15:35Z)
• Quantum Federated Learning with Quantum Data [87.49715898878858]
  Quantum machine learning (QML) has emerged as a promising field that leans on the developments in quantum computing to explore large complex machine learning problems. This paper proposes the first fully quantum federated learning framework that can operate over quantum data and, thus, share the learning of quantum circuit parameters in a decentralized manner.
  arXiv  Detail & Related papers  (2021-05-30T12:19:27Z)
• Tensor Network Quantum Virtual Machine for Simulating Quantum Circuits at Exascale [57.84751206630535]
  We present a modernized version of the Quantum Virtual Machine (TNQVM) which serves as a quantum circuit simulation backend in the e-scale ACCelerator (XACC) framework. The new version is based on the general purpose, scalable network processing library, ExaTN, and provides multiple quantum circuit simulators. By combining the portable XACC quantum processors and the scalable ExaTN backend we introduce an end-to-end virtual development environment which can scale from laptops to future exascale platforms.
  arXiv  Detail & Related papers  (2021-04-21T13:26:42Z)
• Quantum circuit architecture search for variational quantum algorithms [88.71725630554758]
  We propose a resource and runtime efficient scheme termed quantum architecture search (QAS) QAS automatically seeks a near-optimal ansatz to balance benefits and side-effects brought by adding more noisy quantum gates. We implement QAS on both the numerical simulator and real quantum hardware, via the IBM cloud, to accomplish data classification and quantum chemistry tasks.
  arXiv  Detail & Related papers  (2020-10-20T12:06:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.