Quantum-Enhanced Channel Mixing in RWKV Models for Time Series Forecasting

• URL: http://arxiv.org/abs/2505.13524v2
• Date: Sat, 31 May 2025 21:21:06 GMT
• Title: Quantum-Enhanced Channel Mixing in RWKV Models for Time Series Forecasting
• Authors: Chi-Sheng Chen, En-Jui Kuo,
• Abstract summary: Recent advancements in neural sequence modeling have led to architectures such as RWKV, which combine recurrent-style time mixing with feedforward channel mixing to enable efficient long-context processing.<n>In this work, we propose QuantumRWKV, a hybrid quantum-Piece extension of the RWKV model, where the standard feedforward network (FFN) is partially replaced by a variational quantum circuit (VQC)<n>The quantum component is designed to enhance nonlinear representational capacity while preserving end-to-end differentiability via the PennyLane framework.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent advancements in neural sequence modeling have led to architectures such as RWKV, which combine recurrent-style time mixing with feedforward channel mixing to enable efficient long-context processing. In this work, we propose QuantumRWKV, a hybrid quantum-classical extension of the RWKV model, where the standard feedforward network (FFN) is partially replaced by a variational quantum circuit (VQC). The quantum component is designed to enhance nonlinear representational capacity while preserving end-to-end differentiability via the PennyLane framework. To assess the impact of quantum enhancements, we conduct a comparative evaluation between QuantumRWKV and its classical counterpart across ten synthetic time-series forecasting tasks, encompassing linear (ARMA), chaotic (Logistic Map), oscillatory (Damped Oscillator, Sine Wave), and regime-switching signals. Our results show that QuantumRWKV outperforms the classical model in 6 out of 10 tasks, particularly excelling in sequences with nonlinear or chaotic dynamics, such as Chaotic Logistic, Noisy Damped Oscillator, Sine Wave, Triangle Wave, Sawtooth, and ARMA. However, it underperforms on tasks involving sharp regime shifts (Piecewise Regime) or smoother periodic patterns (Damped Oscillator, Seasonal Trend, Square Wave). This study provides one of the first systematic comparisons between hybrid quantum-classical and classical recurrent models in temporal domains, highlighting the scenarios where quantum circuits can offer tangible advantages. We conclude with a discussion on architectural trade-offs, such as variance sensitivity in quantum layers, and outline future directions for scaling quantum integration in long-context temporal learning systems.

Related papers

• Toward Practical Quantum Machine Learning: A Novel Hybrid Quantum LSTM for Fraud Detection [0.1398098625978622]
  We present a novel hybrid quantum-classical neural network architecture for fraud detection.<n>By leveraging quantum phenomena such as superposition and entanglement, our model enhances the feature representation of sequential transaction data.<n>Results demonstrate competitive improvements in accuracy, precision, recall, and F1 score relative to a conventional LSTM baseline.
  arXiv  Detail & Related papers  (2025-04-30T19:09:12Z)
• Quantum Adaptive Self-Attention for Quantum Transformer Models [0.0]
  We propose Quantum Adaptive Self-Attention (QASA), a novel hybrid architecture that enhances classical Transformer models with a quantum attention mechanism.<n>QASA replaces dot-product attention with a parameterized quantum circuit (PQC) that adaptively captures inter-token relationships in the quantum Hilbert space.<n> Experiments on synthetic time-series tasks demonstrate that QASA achieves faster convergence and superior generalization compared to both standard Transformers and reduced classical variants.
  arXiv  Detail & Related papers  (2025-04-05T02:52:37Z)
• Toward Large-Scale Distributed Quantum Long Short-Term Memory with Modular Quantum Computers [5.673361333697935]
  We introduce a Distributed Quantum Long Short-Term Memory (QLSTM) framework to address scalability challenges on Noisy Intermediate-Scale Quantum (NISQ) devices.<n>QLSTM captures long-range temporal dependencies, while a distributed architecture partitions the underlying Variational Quantum Circuits into smaller, manageable subcircuits.<n>We demonstrate that the distributed QLSTM achieves stable convergence and improved training dynamics compared to classical approaches.
  arXiv  Detail & Related papers  (2025-03-18T10:07:34Z)
• Quantum Kernel-Based Long Short-term Memory for Climate Time-Series Forecasting [0.24739484546803336]
  We present the Quantum Kernel-Based Long short-memory (QK-LSTM) network, which integrates quantum kernel methods into classical LSTM architectures.<n>QK-LSTM captures intricate nonlinear dependencies and temporal dynamics with fewer trainable parameters.
  arXiv  Detail & Related papers  (2024-12-12T01:16:52Z)
• Bayesian Quantum Amplitude Estimation [49.1574468325115]
  We introduce BAE, a noise-aware Bayesian algorithm for quantum amplitude estimation.<n>We show that BAE achieves Heisenberg-limited estimation and benchmark it against other approaches.
  arXiv  Detail & Related papers  (2024-12-05T18:09:41Z)
• Fourier Neural Operators for Learning Dynamics in Quantum Spin Systems [77.88054335119074]
  We use FNOs to model the evolution of random quantum spin systems. We apply FNOs to a compact set of Hamiltonian observables instead of the entire $2n$ quantum wavefunction.
  arXiv  Detail & Related papers  (2024-09-05T07:18:09Z)
• Learning to Program Variational Quantum Circuits with Fast Weights [3.6881738506505988]
  This paper introduces the Quantum Fast Weight Programmers (QFWP) as a solution to the temporal or sequential learning challenge. The proposed QFWP model achieves learning of temporal dependencies without necessitating the use of quantum recurrent neural networks. Numerical simulations conducted in this study showcase the efficacy of the proposed QFWP model in both time-series prediction and RL tasks.
  arXiv  Detail & Related papers  (2024-02-27T18:53:18Z)
• Towards Efficient Quantum Hybrid Diffusion Models [68.43405413443175]
  We propose a new methodology to design quantum hybrid diffusion models. We propose two possible hybridization schemes combining quantum computing's superior generalization with classical networks' modularity.
  arXiv  Detail & Related papers  (2024-02-25T16:57:51Z)
• Quantum Effects on the Synchronization Dynamics of the Kuramoto Model [62.997667081978825]
  We show that quantum fluctuations hinder the emergence of synchronization, albeit not entirely suppressing it. We derive an analytical expression for the critical coupling, highlighting its dependence on the model parameters.
  arXiv  Detail & Related papers  (2023-06-16T16:41:16Z)
• Dynamics with autoregressive neural quantum states: application to critical quench dynamics [41.94295877935867]
  We present an alternative general scheme that enables one to capture long-time dynamics of quantum systems in a stable fashion. We apply the scheme to time-dependent quench dynamics by investigating the Kibble-Zurek mechanism in the two-dimensional quantum Ising model.
  arXiv  Detail & Related papers  (2022-09-07T15:50:00Z)
• Out-of-equilibrium dynamics arising from slow round-trip variations of Hamiltonian parameters across quantum and classical critical points [0.0]
  We address the out-of-equilibrium dynamics of many-body systems subject to slow time-dependent round-trip protocols across quantum and classical (thermal) phase transitions. We consider protocols where one relevant parameter w is slowly changed across its critical point wc = 0, linearly in time with a large time scale ts.
  arXiv  Detail & Related papers  (2022-05-17T13:27:32Z)
• Quantum algorithms for quantum dynamics: A performance study on the spin-boson model [68.8204255655161]
  Quantum algorithms for quantum dynamics simulations are traditionally based on implementing a Trotter-approximation of the time-evolution operator. variational quantum algorithms have become an indispensable alternative, enabling small-scale simulations on present-day hardware. We show that, despite providing a clear reduction of quantum gate cost, the variational method in its current implementation is unlikely to lead to a quantum advantage.
  arXiv  Detail & Related papers  (2021-08-09T18:00:05Z)
• Time-Reversal Symmetric ODE Network [138.02741983098454]
  Time-reversal symmetry is a fundamental property that frequently holds in classical and quantum mechanics. We propose a novel loss function that measures how well our ordinary differential equation (ODE) networks comply with this time-reversal symmetry. We show that, even for systems that do not possess the full time-reversal symmetry, TRS-ODENs can achieve better predictive performances over baselines.
  arXiv  Detail & Related papers  (2020-07-22T12:19:40Z)
• Feedback-induced instabilities and dynamics in the Jaynes-Cummings model [62.997667081978825]
  We investigate the coherence and steady-state properties of the Jaynes-Cummings model subjected to time-delayed coherent feedback. The introduced feedback qualitatively modifies the dynamical response and steady-state quantum properties of the system.
  arXiv  Detail & Related papers  (2020-06-20T10:07:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.