Training Hybrid Deep Quantum Neural Network for Efficient Reinforcement Learning

• URL: http://arxiv.org/abs/2503.09119v5
• Date: Wed, 18 Jun 2025 05:29:17 GMT
• Title: Training Hybrid Deep Quantum Neural Network for Efficient Reinforcement Learning
• Authors: Jie Luo, Xueyin Chen, Jeremy Kulcsar, Georgios Korpas, Giulio Giaconi,
• Abstract summary: Quantum circuits embed data in a Hilbert space whose dimensionality grows exponentially with the number of qubits.<n>We introduce qtDNN, a tangential surrogate that locally approximates a quantum circuit.<n>We design hDQNN-TD3, a hybrid deep quantum neural network for continuous-control reinforcement learning.
• Score: 2.2978333459052815
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Quantum circuits embed data in a Hilbert space whose dimensionality grows exponentially with the number of qubits, allowing even shallow parameterised quantum circuits (PQCs) to represent highly-correlated probability distributions that are costly for classical networks to capture. Reinforcement-learning (RL) agents, which must reason over long-horizon, continuous-control tasks, stand to benefit from this expressive quantum feature space, but only if the quantum layers can be trained jointly with the surrounding deep-neural components. Current gradient-estimation techniques (e.g., parameter-shift rule) make such hybrid training impractical for realistic RL workloads, because every gradient step requires a prohibitive number of circuit evaluations and thus erodes the potential quantum advantage. We introduce qtDNN, a tangential surrogate that locally approximates a PQC with a small differentiable network trained on-the-fly from the same minibatch. Embedding qtDNN inside the computation graph yields scalable batch gradients while keeping the original quantum layer for inference. Building on qtDNN we design hDQNN-TD3, a hybrid deep quantum neural network for continuous-control reinforcement learning based on the TD3 architecture. On the high-dimensional Humanoid-v4 benchmark, our agent reaches a test return that surpasses classical TD3, SAC and PPO baselines trained with identical compute. To our knowledge this is the first PQC-enhanced policy that matches or exceeds state-of-the-art classical performance on Humanoid. qtDNN has the potential to reduce quantum-hardware calls significantly and is designed to be compatible with today's NISQ devices. The method opens a path toward applying hybrid quantum models to large-scale RL and other gradient-intensive machine-learning tasks.

Related papers

• Q-Fusion: Diffusing Quantum Circuits [2.348041867134616]
  We propose a diffusion-based algorithm leveraging the LayerDAG framework to generate new quantum circuits. Our results demonstrate that the proposed model consistently generates 100% valid quantum circuit outputs.
  arXiv  Detail & Related papers  (2025-04-29T14:10:10Z)
• Quantum parallel information exchange (QPIE) hybrid network with transfer learning [18.43273756128771]
  Quantum machine learning (QML) has emerged as an innovative framework with the potential to uncover complex patterns. We introduce quantum parallel information exchange (QPIE) hybrid network, a new non-sequential hybrid classical quantum model architecture. We develop a dynamic gradient selection method that applies the parameter shift rule on quantum processing units.
  arXiv  Detail & Related papers  (2025-04-05T17:25:26Z)
• Quantum-Train-Based Distributed Multi-Agent Reinforcement Learning [5.673361333697935]
  Quantum-Train-Based Distributed Multi-Agent Reinforcement Learning (Dist-QTRL) We introduce Quantum-Train-Based Distributed Multi-Agent Reinforcement Learning (Dist-QTRL)
  arXiv  Detail & Related papers  (2024-12-12T00:51:41Z)
• Leveraging Pre-Trained Neural Networks to Enhance Machine Learning with Variational Quantum Circuits [48.33631905972908]
  We introduce an innovative approach that utilizes pre-trained neural networks to enhance Variational Quantum Circuits (VQC) This technique effectively separates approximation error from qubit count and removes the need for restrictive conditions. Our results extend to applications such as human genome analysis, demonstrating the broad applicability of our approach.
  arXiv  Detail & Related papers  (2024-11-13T12:03:39Z)
• Federated Quantum-Train with Batched Parameter Generation [3.697453416360906]
  We introduce the Federated Quantum-Train (QT) framework, which integrates the QT model into federated learning. Our approach significantly reduces qubit usage from 19 to as low as 8 qubits while reducing generalization error.
  arXiv  Detail & Related papers  (2024-09-04T14:39:11Z)
• Efficient Learning for Linear Properties of Bounded-Gate Quantum Circuits [63.733312560668274]
  Given a quantum circuit containing d tunable RZ gates and G-d Clifford gates, can a learner perform purely classical inference to efficiently predict its linear properties? We prove that the sample complexity scaling linearly in d is necessary and sufficient to achieve a small prediction error, while the corresponding computational complexity may scale exponentially in d. We devise a kernel-based learning model capable of trading off prediction error and computational complexity, transitioning from exponential to scaling in many practical settings.
  arXiv  Detail & Related papers  (2024-08-22T08:21:28Z)
• Quantum-Train: Rethinking Hybrid Quantum-Classical Machine Learning in the Model Compression Perspective [7.7063925534143705]
  We introduce the Quantum-Train(QT) framework, a novel approach that integrates quantum computing with machine learning algorithms. QT achieves remarkable results by employing a quantum neural network alongside a classical mapping model.
  arXiv  Detail & Related papers  (2024-05-18T14:35:57Z)
• Multi-GPU-Enabled Hybrid Quantum-Classical Workflow in Quantum-HPC Middleware: Applications in Quantum Simulations [1.9922905420195367]
  This study introduces an innovative distribution-aware Quantum-Classical-Quantum architecture. It integrates cutting-edge quantum software framework works with high-performance classical computing resources. It addresses challenges in quantum simulation for materials and condensed matter physics.
  arXiv  Detail & Related papers  (2024-03-09T07:38:45Z)
• A Quantum-Classical Collaborative Training Architecture Based on Quantum State Fidelity [50.387179833629254]
  We introduce a collaborative classical-quantum architecture called co-TenQu. Co-TenQu enhances a classical deep neural network by up to 41.72% in a fair setting. It outperforms other quantum-based methods by up to 1.9 times and achieves similar accuracy while utilizing 70.59% fewer qubits.
  arXiv  Detail & Related papers  (2024-02-23T14:09:41Z)
• Studying the Impact of Quantum-Specific Hyperparameters on Hybrid Quantum-Classical Neural Networks [4.951980887762045]
  hybrid quantum-classical neural networks (HQNNs) represent a promising solution that combines the strengths of classical machine learning with quantum computing capabilities. In this paper, we investigate the impact of these variations on different HQNN models for image classification tasks, implemented on the PennyLane framework. We aim to uncover intuitive and counter-intuitive learning patterns of HQNN models within granular levels of controlled quantum perturbations, to form a sound basis for their correlation to accuracy and training time.
  arXiv  Detail & Related papers  (2024-02-16T11:44:25Z)
• QuantumSEA: In-Time Sparse Exploration for Noise Adaptive Quantum Circuits [82.50620782471485]
  QuantumSEA is an in-time sparse exploration for noise-adaptive quantum circuits. It aims to achieve two key objectives: (1) implicit circuits capacity during training and (2) noise robustness. Our method establishes state-of-the-art results with only half the number of quantum gates and 2x time saving of circuit executions.
  arXiv  Detail & Related papers  (2024-01-10T22:33:00Z)
• Splitting and Parallelizing of Quantum Convolutional Neural Networks for Learning Translationally Symmetric Data [0.0]
  We propose a novel architecture called split-parallelizing QCNN (sp-QCNN) By splitting the quantum circuit based on translational symmetry, the sp-QCNN can substantially parallelize the conventional QCNN without increasing the number of qubits. We show that the sp-QCNN can achieve comparable classification accuracy to the conventional QCNN while considerably reducing the measurement resources required.
  arXiv  Detail & Related papers  (2023-06-12T18:00:08Z)
• Quafu-RL: The Cloud Quantum Computers based Quantum Reinforcement Learning [0.0]
  In this work, we take the first step towards executing benchmark quantum reinforcement problems on real devices equipped with at most 136 qubits on BAQIS Quafu quantum computing cloud. The experimental results demonstrate that the Reinforcement Learning (RL) agents are capable of achieving goals that are slightly relaxed both during the training and inference stages.
  arXiv  Detail & Related papers  (2023-05-29T09:13:50Z)
• Quantum Imitation Learning [74.15588381240795]
  We propose quantum imitation learning (QIL) with a hope to utilize quantum advantage to speed up IL. We develop two QIL algorithms, quantum behavioural cloning (Q-BC) and quantum generative adversarial imitation learning (Q-GAIL) Experiment results demonstrate that both Q-BC and Q-GAIL can achieve comparable performance compared to classical counterparts.
  arXiv  Detail & Related papers  (2023-04-04T12:47:35Z)
• TeD-Q: a tensor network enhanced distributed hybrid quantum machine learning framework [48.491303218786044]
  TeD-Q is an open-source software framework for quantum machine learning.<n>It seamlessly integrates classical machine learning libraries with quantum simulators.<n>It provides a graphical mode in which the quantum circuit and the training progress can be visualized in real-time.
  arXiv  Detail & Related papers  (2023-01-13T09:35:05Z)
• Synergy Between Quantum Circuits and Tensor Networks: Short-cutting the Race to Practical Quantum Advantage [43.3054117987806]
  We introduce a scalable procedure for harnessing classical computing resources to provide pre-optimized initializations for quantum circuits. We show this method significantly improves the trainability and performance of PQCs on a variety of problems. By demonstrating a means of boosting limited quantum resources using classical computers, our approach illustrates the promise of this synergy between quantum and quantum-inspired models in quantum computing.
  arXiv  Detail & Related papers  (2022-08-29T15:24:03Z)
• Recent Advances for Quantum Neural Networks in Generative Learning [98.88205308106778]
  Quantum generative learning models (QGLMs) may surpass their classical counterparts. We review the current progress of QGLMs from the perspective of machine learning. We discuss the potential applications of QGLMs in both conventional machine learning tasks and quantum physics.
  arXiv  Detail & Related papers  (2022-06-07T07:32:57Z)
• Optimizing Tensor Network Contraction Using Reinforcement Learning [86.05566365115729]
  We propose a Reinforcement Learning (RL) approach combined with Graph Neural Networks (GNN) to address the contraction ordering problem. The problem is extremely challenging due to the huge search space, the heavy-tailed reward distribution, and the challenging credit assignment. We show how a carefully implemented RL-agent that uses a GNN as the basic policy construct can address these challenges.
  arXiv  Detail & Related papers  (2022-04-18T21:45:13Z)
• Quantum Federated Learning with Quantum Data [87.49715898878858]
  Quantum machine learning (QML) has emerged as a promising field that leans on the developments in quantum computing to explore large complex machine learning problems. This paper proposes the first fully quantum federated learning framework that can operate over quantum data and, thus, share the learning of quantum circuit parameters in a decentralized manner.
  arXiv  Detail & Related papers  (2021-05-30T12:19:27Z)
• Tensor Network Quantum Virtual Machine for Simulating Quantum Circuits at Exascale [57.84751206630535]
  We present a modernized version of the Quantum Virtual Machine (TNQVM) which serves as a quantum circuit simulation backend in the e-scale ACCelerator (XACC) framework. The new version is based on the general purpose, scalable network processing library, ExaTN, and provides multiple quantum circuit simulators. By combining the portable XACC quantum processors and the scalable ExaTN backend we introduce an end-to-end virtual development environment which can scale from laptops to future exascale platforms.
  arXiv  Detail & Related papers  (2021-04-21T13:26:42Z)
• Branching Quantum Convolutional Neural Networks [0.0]
  Small-scale quantum computers are already showing potential gains in learning tasks on large quantum and very large classical data sets. We present a generalization of QCNN, the branching quantum convolutional neural network, or bQCNN, with substantially higher expressibility.
  arXiv  Detail & Related papers  (2020-12-28T19:00:03Z)
• Hybrid quantum-classical classifier based on tensor network and variational quantum circuit [0.0]
  We introduce a hybrid model combining the quantum-inspired tensor networks (TN) and the variational quantum circuits (VQC) to perform supervised learning tasks. We show that a matrix product state based TN with low bond dimensions performs better than PCA as a feature extractor to compress data for the input of VQCs in the binary classification of MNIST dataset.
  arXiv  Detail & Related papers  (2020-11-30T09:43:59Z)
• On the learnability of quantum neural networks [132.1981461292324]
  We consider the learnability of the quantum neural network (QNN) built on the variational hybrid quantum-classical scheme. We show that if a concept can be efficiently learned by QNN, then it can also be effectively learned by QNN even with gate noise.
  arXiv  Detail & Related papers  (2020-07-24T06:34:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.