Accelerating Quantum Reinforcement Learning with a Quantum Natural Policy Gradient Based Approach

• URL: http://arxiv.org/abs/2501.16243v1
• Date: Mon, 27 Jan 2025 17:38:30 GMT
• Title: Accelerating Quantum Reinforcement Learning with a Quantum Natural Policy Gradient Based Approach
• Authors: Yang Xu, Vaneet Aggarwal,
• Abstract summary: This paper introduces a Quantum Natural Policy Gradient (QNPG) algorithm, which replaces the random sampling used in classicalNPG estimators with a deterministic gradient estimation approach. The proposed QNPG algorithm achieves a sample complexity of $tildemathcalO(epsilon-1.5)$ for queries to the quantum oracle, significantly improving the classical lower bound of $tildemathcalO(epsilon-2)$ for queries to the Markov Decision Process (MDP)
• Score: 36.05085942729295
• License:
• Abstract: We address the problem of quantum reinforcement learning (QRL) under model-free settings with quantum oracle access to the Markov Decision Process (MDP). This paper introduces a Quantum Natural Policy Gradient (QNPG) algorithm, which replaces the random sampling used in classical Natural Policy Gradient (NPG) estimators with a deterministic gradient estimation approach, enabling seamless integration into quantum systems. While this modification introduces a bounded bias in the estimator, the bias decays exponentially with increasing truncation levels. This paper demonstrates that the proposed QNPG algorithm achieves a sample complexity of $\tilde{\mathcal{O}}(\epsilon^{-1.5})$ for queries to the quantum oracle, significantly improving the classical lower bound of $\tilde{\mathcal{O}}(\epsilon^{-2})$ for queries to the MDP.

Related papers

• Quantum Policy Gradient in Reproducing Kernel Hilbert Space [3.8916312075738273]
  Parametrised quantum circuits offer expressive and data-efficient representations for machine learning. The representation of quantum circuits in terms of quantum kernels has been studied widely in quantum supervised learning. This paper proposes parametric and non-parametric policy gradient and actor-critic algorithms with quantum kernel policies in quantum environments.
  arXiv  Detail & Related papers  (2024-11-11T01:34:10Z)
• Projective Quantum Eigensolver with Generalized Operators [0.0]
  We develop a methodology for determining the generalized operators in terms of a closed form residual equations in the PQE framework. With the application on several molecular systems, we have demonstrated our ansatz achieves similar accuracy to the (disentangled) UCC with singles, doubles and triples.
  arXiv  Detail & Related papers  (2024-10-21T15:40:22Z)
• Quantum Natural Stochastic Pairwise Coordinate Descent [6.187270874122921]
  Quantum machine learning through variational quantum algorithms (VQAs) has gained substantial attention in recent years. This paper introduces the quantum natural pairwise coordinate descent (2QNSCD) optimization method. We develop a highly sparse unbiased estimator of the novel metric tensor using a quantum circuit with gate complexity $Theta(1)$ times that of the parameterized quantum circuit and single-shot quantum measurements.
  arXiv  Detail & Related papers  (2024-07-18T18:57:29Z)
• GRAPE optimization for open quantum systems with time-dependent decoherence rates driven by coherent and incoherent controls [77.34726150561087]
  The GRadient Ascent Pulse Engineering (GRAPE) method is widely used for optimization in quantum control. We adopt GRAPE method for optimizing objective functionals for open quantum systems driven by both coherent and incoherent controls. The efficiency of the algorithm is demonstrated through numerical simulations for the state-to-state transition problem.
  arXiv  Detail & Related papers  (2023-07-17T13:37:18Z)
• Quantum Annealing for Single Image Super-Resolution [86.69338893753886]
  We propose a quantum computing-based algorithm to solve the single image super-resolution (SISR) problem. The proposed AQC-based algorithm is demonstrated to achieve improved speed-up over a classical analog while maintaining comparable SISR accuracy.
  arXiv  Detail & Related papers  (2023-04-18T11:57:15Z)
• Quantum Gate Generation in Two-Level Open Quantum Systems by Coherent and Incoherent Photons Found with Gradient Search [77.34726150561087]
  We consider an environment formed by incoherent photons as a resource for controlling open quantum systems via an incoherent control. We exploit a coherent control in the Hamiltonian and an incoherent control in the dissipator which induces the time-dependent decoherence rates.
  arXiv  Detail & Related papers  (2023-02-28T07:36:02Z)
• Quantum algorithms for quantum dynamics: A performance study on the spin-boson model [68.8204255655161]
  Quantum algorithms for quantum dynamics simulations are traditionally based on implementing a Trotter-approximation of the time-evolution operator. variational quantum algorithms have become an indispensable alternative, enabling small-scale simulations on present-day hardware. We show that, despite providing a clear reduction of quantum gate cost, the variational method in its current implementation is unlikely to lead to a quantum advantage.
  arXiv  Detail & Related papers  (2021-08-09T18:00:05Z)
• Q-Match: Iterative Shape Matching via Quantum Annealing [64.74942589569596]
  Finding shape correspondences can be formulated as an NP-hard quadratic assignment problem (QAP) This paper proposes Q-Match, a new iterative quantum method for QAPs inspired by the alpha-expansion algorithm. Q-Match can be applied for shape matching problems iteratively, on a subset of well-chosen correspondences, allowing us to scale to real-world problems.
  arXiv  Detail & Related papers  (2021-05-06T17:59:38Z)
• Hybrid quantum variational algorithm for simulating open quantum systems with near-term devices [0.0]
  Hybrid quantum-classical (HQC) algorithms make it possible to use near-term quantum devices supported by classical computational resources. We develop an HQC algorithm using an efficient variational optimization approach to simulate open system dynamics.
  arXiv  Detail & Related papers  (2020-08-12T13:49:29Z)
• Measuring Analytic Gradients of General Quantum Evolution with the Stochastic Parameter Shift Rule [0.0]
  We study the problem of estimating the gradient of the function to be optimized directly from quantum measurements. We derive a mathematically exact formula that provides an algorithm for estimating the gradient of any multi-qubit parametric quantum evolution. Our algorithm continues to work, although with some approximations, even when all the available quantum gates are noisy.
  arXiv  Detail & Related papers  (2020-05-20T18:24:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.