Learning with SASQuaTCh: a Novel Variational Quantum Transformer Architecture with Kernel-Based Self-Attention

• URL: http://arxiv.org/abs/2403.14753v1
• Date: Thu, 21 Mar 2024 18:00:04 GMT
• Title: Learning with SASQuaTCh: a Novel Variational Quantum Transformer Architecture with Kernel-Based Self-Attention
• Authors: Ethan N. Evans, Matthew Cook, Zachary P. Bradshaw, Margarite L. LaBorde,
• Abstract summary: We show that quantum circuits can efficiently express a self-attention mechanism through the perspective of kernel-based operator learning. In this work, we are able to represent deep layers of a vision transformer network using simple gate operations and a set of multi-dimensional quantum Fourier transforms. We analyze our novel variational quantum circuit, which we call Self-Attention Sequential Quantum Transformer Channel (SASTQuaCh), and demonstrate its utility on simplified classification problems.
• Score: 0.464982780843177
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The widely popular transformer network popularized by the generative pre-trained transformer (GPT) has a large field of applicability, including predicting text and images, classification, and even predicting solutions to the dynamics of physical systems. In the latter context, the continuous analog of the self-attention mechanism at the heart of transformer networks has been applied to learning the solutions of partial differential equations and reveals a convolution kernel nature that can be exploited by the Fourier transform. It is well known that many quantum algorithms that have provably demonstrated a speedup over classical algorithms utilize the quantum Fourier transform. In this work, we explore quantum circuits that can efficiently express a self-attention mechanism through the perspective of kernel-based operator learning. In this perspective, we are able to represent deep layers of a vision transformer network using simple gate operations and a set of multi-dimensional quantum Fourier transforms. We analyze the computational and parameter complexity of our novel variational quantum circuit, which we call Self-Attention Sequential Quantum Transformer Channel (SASQuaTCh), and demonstrate its utility on simplified classification problems.

Related papers

• Exploiting recursive structures for the design of novel quantum primitives [0.1227734309612871]
  This paper focuses on generating novel quantum primitives. We show how these structures can be exploited to design new, potentially advantageous quantum algorithms. We comment on the potential impact on quantum algorithms, numerical analysis, and signal processing.
  arXiv  Detail & Related papers  (2024-10-17T17:45:50Z)
• Efficient Learning for Linear Properties of Bounded-Gate Quantum Circuits [63.733312560668274]
  Given a quantum circuit containing d tunable RZ gates and G-d Clifford gates, can a learner perform purely classical inference to efficiently predict its linear properties? We prove that the sample complexity scaling linearly in d is necessary and sufficient to achieve a small prediction error, while the corresponding computational complexity may scale exponentially in d. We devise a kernel-based learning model capable of trading off prediction error and computational complexity, transitioning from exponential to scaling in many practical settings.
  arXiv  Detail & Related papers  (2024-08-22T08:21:28Z)
• Zero and Finite Temperature Quantum Simulations Powered by Quantum Magic [1.5998200006932823]
  We introduce a quantum information theory-inspired method to improve the characterization of many-body Hamiltonians on near-term quantum devices. We show that our protocol leads to significant performance improvements for zero and finite temperature free energy calculations on both digital and analog quantum hardware.
  arXiv  Detail & Related papers  (2023-08-22T17:59:41Z)
• TeD-Q: a tensor network enhanced distributed hybrid quantum machine learning framework [59.07246314484875]
  TeD-Q is an open-source software framework for quantum machine learning. It seamlessly integrates classical machine learning libraries with quantum simulators. It provides a graphical mode in which the quantum circuit and the training progress can be visualized in real-time.
  arXiv  Detail & Related papers  (2023-01-13T09:35:05Z)
• The Quantum Path Kernel: a Generalized Quantum Neural Tangent Kernel for Deep Quantum Machine Learning [52.77024349608834]
  Building a quantum analog of classical deep neural networks represents a fundamental challenge in quantum computing. Key issue is how to address the inherent non-linearity of classical deep learning. We introduce the Quantum Path Kernel, a formulation of quantum machine learning capable of replicating those aspects of deep machine learning.
  arXiv  Detail & Related papers  (2022-12-22T16:06:24Z)
• Fock-space Schrieffer--Wolff transformation: classically-assisted rank-reduced quantum phase estimation algorithm [0.0]
  In this paper, we focus on the Schrieffer--Wolff (SW) transformation of the electronic Hamiltonians for molecular systems. We demonstrate that by employing Fock-space variants of the SW transformation one can significantly increase the locality of the qubit-mapped similarity transformed Hamiltonians. The RRST formalism serves as a design principle for developing new classes of approximate schemes that reduce the complexity of quantum circuits.
  arXiv  Detail & Related papers  (2022-11-18T23:06:57Z)
• Quantum Phase Processing and its Applications in Estimating Phase and Entropies [10.8525801756287]
  "quantum phase processing" can directly apply arbitrary trigonometric transformations to eigenphases of a unitary operator. Quantum phase processing can extract the eigen-information of quantum systems by simply measuring the ancilla qubit. We propose a new quantum phase estimation algorithm without quantum Fourier transform, which requires the fewest ancilla qubits and matches the best performance so far.
  arXiv  Detail & Related papers  (2022-09-28T17:41:19Z)
• Quantum algorithms for quantum dynamics: A performance study on the spin-boson model [68.8204255655161]
  Quantum algorithms for quantum dynamics simulations are traditionally based on implementing a Trotter-approximation of the time-evolution operator. variational quantum algorithms have become an indispensable alternative, enabling small-scale simulations on present-day hardware. We show that, despite providing a clear reduction of quantum gate cost, the variational method in its current implementation is unlikely to lead to a quantum advantage.
  arXiv  Detail & Related papers  (2021-08-09T18:00:05Z)
• Information Scrambling in Computationally Complex Quantum Circuits [56.22772134614514]
  We experimentally investigate the dynamics of quantum scrambling on a 53-qubit quantum processor. We show that while operator spreading is captured by an efficient classical model, operator entanglement requires exponentially scaled computational resources to simulate.
  arXiv  Detail & Related papers  (2021-01-21T22:18:49Z)
• Experimental Quantum Generative Adversarial Networks for Image Generation [93.06926114985761]
  We experimentally achieve the learning and generation of real-world hand-written digit images on a superconducting quantum processor. Our work provides guidance for developing advanced quantum generative models on near-term quantum devices.
  arXiv  Detail & Related papers  (2020-10-13T06:57:17Z)
• Autoregressive Transformer Neural Network for Simulating Open Quantum Systems via a Probabilistic Formulation [5.668795025564699]
  We present an approach for tackling open quantum system dynamics. We compactly represent quantum states with autoregressive transformer neural networks. Efficient algorithms have been developed to simulate the dynamics of the Liouvillian superoperator.
  arXiv  Detail & Related papers  (2020-09-11T18:00:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.