QLAM: A Quantum Long-Attention Memory Approach to Long-Sequence Token Modeling

• URL: http://arxiv.org/abs/2605.13833v1
• Date: Wed, 13 May 2026 17:56:20 GMT
• Title: QLAM: A Quantum Long-Attention Memory Approach to Long-Sequence Token Modeling
• Abstract summary: We introduce one of the first studies to leverage the superposition property of quantum systems to enhance state-based sequence modeling.<n>QLAM represents the hidden state as a quantum state whose amplitudes encode a superposition of historical information.<n>We evaluate QLAM on sequential variants of standard image classification benchmarks, including sMNIST, sFashion-MNIST, and sCIFAR-10.
• Score: 14.229084182282607
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Modeling long-range dependencies in sequential data remains a central challenge in machine learning. Transformers address this challenge through attention mechanisms, but their quadratic complexity with respect to sequence length limits scalability to long contexts. State-space models (SSMs) provide an efficient alternative with linear-time computation by evolving a latent state through recurrent updates, but their memory is typically formed via additive or linear transitions, which can limit their ability to capture complex global interactions across tokens. In this work, we introduce one of the first studies to leverage the superposition property of quantum systems to enhance state-based sequence modeling. In particular, we propose Quantum Long-Attention Memory (QLAM), a hybrid quantum-classical memory mechanism that can be viewed as a quantum extension of state-space models. Instead of maintaining a classical latent state updated through additive dynamics, QLAM represents the hidden state as a quantum state whose amplitudes encode a superposition of historical information. The state evolves through parameterized quantum circuits conditioned on the input, enabling a non-classical, globally update mechanism. In this way, QLAM preserves the recurrent and linear-time structure of SSMs while fundamentally enriching the memory representation through quantum superposition. Unlike attention mechanisms that explicitly compute pairwise interactions, QLAM implicitly captures global dependencies through the evolution of the quantum state, and retrieves task-relevant information via query-dependent measurements. We evaluate QLAM on sequential variants of standard image classification benchmarks, including sMNIST, sFashion-MNIST, and sCIFAR-10, where images are flattened into token sequences. Across all tasks, QLAM consistently improves over recurrent baselines and transformer-based models.

Related papers

• Quantum Dynamic Time Warping for Multivariate Time Series Classification [41.99844472131922]
  We propose a hybrid Quantum Dynamic Time Warping architecture, replacing the classical distance metric with the parameterized geometry of a quantum Hilbert space.<n>We demonstrate this architecture allows untrained quantum kernels to act as highly expressive baselines, while parameterized training effectively untangles deeply overlapping hyper-dimensional data.
  arXiv  Detail & Related papers  (2026-06-26T07:58:37Z)
• Hybrid Quantum-Classical Recurrent Neural Networks [0.10152838128195468]
  We present a hybrid quantum-classical recurrent neural network architecture.<n>The hidden state is the quantum state of an $n$-qubit quantum circuit (PQC) controlled by a classical feedforward network.<n>We evaluate the model in simulation with up to 14 qubits on sentiment analysis, MNIST, permuted MNIST, copying memory, and language modeling.
  arXiv  Detail & Related papers  (2025-10-29T14:21:49Z)
• Towards Quantum Enhanced Adversarial Robustness with Rydberg Reservoir Learning [45.92935470813908]
  Quantum computing reservoir (QRC) leverages the high-dimensional, nonlinear dynamics inherent in quantum many-body systems.<n>Recent studies indicate that perturbation quantums based on variational circuits remain susceptible to adversarials.<n>We investigate the first systematic evaluation of adversarial robustness in a QR based learning model.
  arXiv  Detail & Related papers  (2025-10-15T12:17:23Z)
• Entanglement and Classical Simulability in Quantum Extreme Learning Machines [0.0]
  We investigate Quantum Extreme Learning Machines (QELMs), a quantum analogue of classical Extreme Learning Machines.<n>Our architecture combines dimensionality reduction (via PCA or Autoencoders), quantum state encoding, evolution under an XX Hamiltonian, and projective measurement.<n>We show that this performance enhancement correlates with the onset of entanglement, which improves the embedding of classical data.
  arXiv  Detail & Related papers  (2025-09-08T16:43:37Z)
• Quantum-Optimized Selective State Space Model for Efficient Time Series Prediction [39.146761527401424]
  We propose a hybrid quantum-optimized approach that integrates state space dynamics with a variational quantum gate.<n>We empirically validate Q-SSM on three widely used benchmarks, i.e., ETT, Traffic, and Exchange Rate.<n>Results show that Q-SSM consistently improves over strong baselines.
  arXiv  Detail & Related papers  (2025-08-29T22:00:48Z)
• On the emergence of quantum memory in non-Markovian dynamics [41.94295877935867]
  Non-Markovian dynamics (with memory) is typical in practice, with memory effects being harnessed as a resource for many tasks like quantum error correction and information processing.<n>Yet, the type of memory, classical or quantum, necessary to realize the dynamics of many collision models is not known.<n>In this work, we extend the quantum homogenizer to the non-Markovian regime by introducing intra-ancilla interactions mediated by Fredkin gates, and study the nature of its memory.
  arXiv  Detail & Related papers  (2025-07-29T15:19:26Z)
• Sequential-Parallel Duality in Prefix Scannable Models [68.39855814099997]
  Recent developments have given rise to various models, such as Gated Linear Attention (GLA) and Mamba.<n>This raises a natural question: can we characterize the full class of neural sequence models that support near-constant-time parallel evaluation and linear-time, constant-space sequential inference?
  arXiv  Detail & Related papers  (2025-06-12T17:32:02Z)
• Dynamical learning and quantum memory with non-Hermitian many-body systems [37.69303106863453]
  Non-Hermitian (NH) systems provide a fertile platform for quantum technologies.<n>We investigate this relationship using the example of an interacting NH spin system defined on random graphs.<n>We show that the onset of the first exceptional point corresponds to an abrupt change in the system's learning capacity.
  arXiv  Detail & Related papers  (2025-06-09T11:58:40Z)
• QAMA: Scalable Quantum Annealing Multi-Head Attention Operator for Deep Learning [48.12231190677108]
  Quantum Annealing Multi-Head Attention (QAMA) is proposed, a novel drop-in operator that reformulates attention as an energy-based Hamiltonian optimization problem.<n>In this framework, token interactions are encoded into binary quadratic terms, and quantum annealing is employed to search for low-energy configurations.<n> Empirically, evaluation on both natural language and vision benchmarks shows that, across tasks, accuracy deviates by at most 2.7 points from standard multi-head attention.
  arXiv  Detail & Related papers  (2025-04-15T11:29:09Z)
• Quantum Adaptive Self-Attention for Quantum Transformer Models [0.0]
  We propose Quantum Adaptive Self-Attention (QASA), a novel hybrid architecture that enhances classical Transformer models with a quantum attention mechanism.<n>QASA replaces dot-product attention with a parameterized quantum circuit (PQC) that adaptively captures inter-token relationships in the quantum Hilbert space.<n> Experiments on synthetic time-series tasks demonstrate that QASA achieves faster convergence and superior generalization compared to both standard Transformers and reduced classical variants.
  arXiv  Detail & Related papers  (2025-04-05T02:52:37Z)
• Ab-initio variational wave functions for the time-dependent many-electron Schrödinger equation [41.94295877935867]
  We introduce a variational approach for fermionic time-dependent wave functions, surpassing mean-field approximations. We use time-dependent Jastrow factors and backflow transformations, which are enhanced through neural networks parameterizations. The results showcase the ability of our variational approach to accurately capture the time evolution, providing insight into the quantum dynamics of interacting electronic systems.
  arXiv  Detail & Related papers  (2024-03-12T09:37:22Z)
• QSAN: A Near-term Achievable Quantum Self-Attention Network [73.15524926159702]
  Self-Attention Mechanism (SAM) is good at capturing the internal connections of features. A novel Quantum Self-Attention Network (QSAN) is proposed for image classification tasks on near-term quantum devices.
  arXiv  Detail & Related papers  (2022-07-14T12:22:51Z)
• Quantum associative memory with a single driven-dissipative nonlinear oscillator [0.0]
  We propose a realization of associative memory with a single driven-dissipative quantum oscillator. The model can improve the storage capacity of discrete neuron-based systems in a large regime. We show that the associative-memory capacity is inherently related to the existence of a spectral gap in the Liouvillian superoperator.
  arXiv  Detail & Related papers  (2022-05-19T12:00:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.