Control of Continuous Quantum Systems with Many Degrees of Freedom based
on Convergent Reinforcement Learning
- URL: http://arxiv.org/abs/2212.10705v1
- Date: Wed, 21 Dec 2022 00:52:43 GMT
- Title: Control of Continuous Quantum Systems with Many Degrees of Freedom based
on Convergent Reinforcement Learning
- Authors: Zhikang Wang
- Abstract summary: In this dissertation, we investigate the non-convergence issue of Q-learning.
We develop a new convergent Q-learning algorithm, which we call the convergent deep Q network (C-DQN) algorithm.
We prove the convergence of C-DQN and apply it to the Atari 2600 benchmark.
- Score: 1.8710230264817362
- License: http://creativecommons.org/publicdomain/zero/1.0/
- Abstract: With the development of experimental quantum technology, quantum control has
attracted increasing attention due to the realization of controllable
artificial quantum systems. However, because quantum-mechanical systems are
often too difficult to analytically deal with, heuristic strategies and
numerical algorithms which search for proper control protocols are adopted,
and, deep learning, especially deep reinforcement learning (RL), is a promising
generic candidate solution for the control problems. Although there have been a
few successful applications of deep RL to quantum control problems, most of the
existing RL algorithms suffer from instabilities and unsatisfactory
reproducibility, and require a large amount of fine-tuning and a large
computational budget, both of which limit their applicability. To resolve the
issue of instabilities, in this dissertation, we investigate the
non-convergence issue of Q-learning. Then, we investigate the weakness of
existing convergent approaches that have been proposed, and we develop a new
convergent Q-learning algorithm, which we call the convergent deep Q network
(C-DQN) algorithm, as an alternative to the conventional deep Q network (DQN)
algorithm. We prove the convergence of C-DQN and apply it to the Atari 2600
benchmark. We show that when DQN fail, C-DQN still learns successfully. Then,
we apply the algorithm to the measurement-feedback cooling problems of a
quantum quartic oscillator and a trapped quantum rigid body. We establish the
physical models and analyse their properties, and we show that although both
C-DQN and DQN can learn to cool the systems, C-DQN tends to behave more stably,
and when DQN suffers from instabilities, C-DQN can achieve a better
performance. As the performance of DQN can have a large variance and lack
consistency, C-DQN can be a better choice for researches on complicated control
problems.
Related papers
- Reinforcement learning-assisted quantum architecture search for variational quantum algorithms [0.0]
This thesis focuses on identifying functional quantum circuits in noisy quantum hardware.
We introduce a tensor-based quantum circuit encoding, restrictions on environment dynamics to explore the search space of possible circuits efficiently.
In dealing with various VQAs, our RL-based QAS outperforms existing QAS.
arXiv Detail & Related papers (2024-02-21T12:30:39Z) - Matching Game for Optimized Association in Quantum Communication
Networks [65.16483325184237]
This paper proposes a swap-stable request-QS association algorithm for quantum switches.
It achieves a near-optimal (within 5%) performance in terms of the percentage of served requests.
It is shown to be scalable and maintain its near-optimal performance even when the size of the QCN increases.
arXiv Detail & Related papers (2023-05-22T03:39:18Z) - QNEAT: Natural Evolution of Variational Quantum Circuit Architecture [95.29334926638462]
We focus on variational quantum circuits (VQC), which emerged as the most promising candidates for the quantum counterpart of neural networks.
Although showing promising results, VQCs can be hard to train because of different issues, e.g., barren plateau, periodicity of the weights, or choice of architecture.
We propose a gradient-free algorithm inspired by natural evolution to optimize both the weights and the architecture of the VQC.
arXiv Detail & Related papers (2023-04-14T08:03:20Z) - Quantum Imitation Learning [74.15588381240795]
We propose quantum imitation learning (QIL) with a hope to utilize quantum advantage to speed up IL.
We develop two QIL algorithms, quantum behavioural cloning (Q-BC) and quantum generative adversarial imitation learning (Q-GAIL)
Experiment results demonstrate that both Q-BC and Q-GAIL can achieve comparable performance compared to classical counterparts.
arXiv Detail & Related papers (2023-04-04T12:47:35Z) - Optimizing Tensor Network Contraction Using Reinforcement Learning [86.05566365115729]
We propose a Reinforcement Learning (RL) approach combined with Graph Neural Networks (GNN) to address the contraction ordering problem.
The problem is extremely challenging due to the huge search space, the heavy-tailed reward distribution, and the challenging credit assignment.
We show how a carefully implemented RL-agent that uses a GNN as the basic policy construct can address these challenges.
arXiv Detail & Related papers (2022-04-18T21:45:13Z) - Uncovering Instabilities in Variational-Quantum Deep Q-Networks [0.0]
We show that variational quantum deep Q-networks (VQ-DQN) are subject to instabilities that cause the learned policy to diverge.
We execute RL algorithms on an actual quantum processing unit (an IBM Quantum Device) and investigate differences in behaviour between simulated and physical quantum systems.
arXiv Detail & Related papers (2022-02-10T17:52:44Z) - Supervisory Control of Quantum Discrete Event Systems [2.3097706741644686]
This paper establishes a basic framework of QDES by using it quantum finite automata (QFA) as the modelling formalisms.
We present a number of new examples of QFA to illustrate the supervisory control of QDES and to verify the essential advantages of QDES over classical DES in state complexity.
arXiv Detail & Related papers (2021-04-20T04:17:41Z) - Quantum agents in the Gym: a variational quantum algorithm for deep
Q-learning [0.0]
We introduce a training method for parametrized quantum circuits (PQCs) that can be used to solve RL tasks for discrete and continuous state spaces.
We investigate which architectural choices for quantum Q-learning agents are most important for successfully solving certain types of environments.
arXiv Detail & Related papers (2021-03-28T08:57:22Z) - Quantum circuit architecture search for variational quantum algorithms [88.71725630554758]
We propose a resource and runtime efficient scheme termed quantum architecture search (QAS)
QAS automatically seeks a near-optimal ansatz to balance benefits and side-effects brought by adding more noisy quantum gates.
We implement QAS on both the numerical simulator and real quantum hardware, via the IBM cloud, to accomplish data classification and quantum chemistry tasks.
arXiv Detail & Related papers (2020-10-20T12:06:27Z) - On the learnability of quantum neural networks [132.1981461292324]
We consider the learnability of the quantum neural network (QNN) built on the variational hybrid quantum-classical scheme.
We show that if a concept can be efficiently learned by QNN, then it can also be effectively learned by QNN even with gate noise.
arXiv Detail & Related papers (2020-07-24T06:34:34Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.