Variational quantum policies for reinforcement learning
- URL: http://arxiv.org/abs/2103.05577v1
- Date: Tue, 9 Mar 2021 17:33:09 GMT
- Title: Variational quantum policies for reinforcement learning
- Authors: Sofiene Jerbi, Casper Gyurik, Simon Marshall, Hans J. Briegel, Vedran
Dunjko
- Abstract summary: Variational quantum circuits have recently gained popularity as quantum machine learning models.
In this work, we investigate how to construct and train reinforcement learning policies based on variational quantum circuits.
- Score: 0.0
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Variational quantum circuits have recently gained popularity as quantum
machine learning models. While considerable effort has been invested to train
them in supervised and unsupervised learning settings, relatively little
attention has been given to their potential use in reinforcement learning. In
this work, we leverage the understanding of quantum policy gradient algorithms
in a number of ways. First, we investigate how to construct and train
reinforcement learning policies based on variational quantum circuits. We
propose several designs for quantum policies, provide their learning
algorithms, and test their performance on classical benchmarking environments.
Second, we show the existence of task environments with a provable separation
in performance between quantum learning agents and any polynomial-time
classical learner, conditioned on the widely-believed classical hardness of the
discrete logarithm problem. We also consider more natural settings, in which we
show an empirical quantum advantage of our quantum policies over standard
neural-network policies. Our results constitute a first step towards
establishing a practical near-term quantum advantage in a reinforcement
learning setting. Additionally, we believe that some of our design choices for
variational quantum policies may also be beneficial to other models based on
variational quantum circuits, such as quantum classifiers and quantum
regression models.
Related papers
- Separable Power of Classical and Quantum Learning Protocols Through the Lens of No-Free-Lunch Theorem [70.42372213666553]
The No-Free-Lunch (NFL) theorem quantifies problem- and data-independent generalization errors regardless of the optimization process.
We categorize a diverse array of quantum learning algorithms into three learning protocols designed for learning quantum dynamics under a specified observable.
Our derived NFL theorems demonstrate quadratic reductions in sample complexity across CLC-LPs, ReQu-LPs, and Qu-LPs.
We attribute this performance discrepancy to the unique capacity of quantum-related learning protocols to indirectly utilize information concerning the global phases of non-orthogonal quantum states.
arXiv Detail & Related papers (2024-05-12T09:05:13Z) - Information-theoretic generalization bounds for learning from quantum data [5.0739329301140845]
We propose a general mathematical formalism for describing quantum learning by training on classical-quantum data.
We prove bounds on the expected generalization error of a quantum learner in terms of classical and quantum information-theoretic quantities.
Our work lays a foundation for a unifying quantum information-theoretic perspective on quantum learning.
arXiv Detail & Related papers (2023-11-09T17:21:38Z) - Quantum data learning for quantum simulations in high-energy physics [55.41644538483948]
We explore the applicability of quantum-data learning to practical problems in high-energy physics.
We make use of ansatz based on quantum convolutional neural networks and numerically show that it is capable of recognizing quantum phases of ground states.
The observation of non-trivial learning properties demonstrated in these benchmarks will motivate further exploration of the quantum-data learning architecture in high-energy physics.
arXiv Detail & Related papers (2023-06-29T18:00:01Z) - Quantum Imitation Learning [74.15588381240795]
We propose quantum imitation learning (QIL) with a hope to utilize quantum advantage to speed up IL.
We develop two QIL algorithms, quantum behavioural cloning (Q-BC) and quantum generative adversarial imitation learning (Q-GAIL)
Experiment results demonstrate that both Q-BC and Q-GAIL can achieve comparable performance compared to classical counterparts.
arXiv Detail & Related papers (2023-04-04T12:47:35Z) - Quantum Deep Hedging [10.243020478772056]
We look at the problem of hedging where deep reinforcement learning offers a powerful framework for real markets.
We develop quantum reinforcement learning methods based on policy-search and distributional actor-critic algorithms.
We successfully implement the proposed models on a trapped-ion quantum processor.
arXiv Detail & Related papers (2023-03-29T10:42:50Z) - Quantum Machine Learning: from physics to software engineering [58.720142291102135]
We show how classical machine learning approach can help improve the facilities of quantum computers.
We discuss how quantum algorithms and quantum computers may be useful for solving classical machine learning tasks.
arXiv Detail & Related papers (2023-01-04T23:37:45Z) - The Quantum Path Kernel: a Generalized Quantum Neural Tangent Kernel for
Deep Quantum Machine Learning [52.77024349608834]
Building a quantum analog of classical deep neural networks represents a fundamental challenge in quantum computing.
Key issue is how to address the inherent non-linearity of classical deep learning.
We introduce the Quantum Path Kernel, a formulation of quantum machine learning capable of replicating those aspects of deep machine learning.
arXiv Detail & Related papers (2022-12-22T16:06:24Z) - Quantum policy gradient algorithms [1.5293427903448025]
We show that speed-ups in learning are possible when given quantum access to reinforcement learning environments.
In this work, we design quantum algorithms to train state-of-the-art reinforcement learning policies.
We find that reinforcement learning policies derived from parametrized quantum circuits are well-behaved.
arXiv Detail & Related papers (2022-12-19T09:45:58Z) - Time-Optimal Quantum Driving by Variational Circuit Learning [2.9582851733261286]
Digital quantum simulation and hybrid circuit learning opens up new prospects for quantum optimal control.
We simulate the wave-packet expansion of a trapped quantum particle on a quantum device with a finite number qubits.
We discuss the robustness of our method against errors and demonstrate the absence of barren plateaus in the circuit.
arXiv Detail & Related papers (2022-11-01T11:53:49Z) - Variational Quantum Soft Actor-Critic [1.90365714903665]
We develop a quantum reinforcement learning algorithm based on soft actor-critic -- one of the state-of-the-art methods for continuous control.
We show that this quantum version of soft actor-critic is comparable with the original soft actor-critic, using much less adjustable parameters.
arXiv Detail & Related papers (2021-12-20T06:31:06Z) - Quantum algorithms for quantum dynamics: A performance study on the
spin-boson model [68.8204255655161]
Quantum algorithms for quantum dynamics simulations are traditionally based on implementing a Trotter-approximation of the time-evolution operator.
variational quantum algorithms have become an indispensable alternative, enabling small-scale simulations on present-day hardware.
We show that, despite providing a clear reduction of quantum gate cost, the variational method in its current implementation is unlikely to lead to a quantum advantage.
arXiv Detail & Related papers (2021-08-09T18:00:05Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.