Variational Quantum Soft Actor-Critic

• URL: http://arxiv.org/abs/2112.11921v1
• Date: Mon, 20 Dec 2021 06:31:06 GMT
• Title: Variational Quantum Soft Actor-Critic
• Authors: Qingfeng Lan
• Abstract summary: We develop a quantum reinforcement learning algorithm based on soft actor-critic -- one of the state-of-the-art methods for continuous control. We show that this quantum version of soft actor-critic is comparable with the original soft actor-critic, using much less adjustable parameters.
• Score: 1.90365714903665
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Quantum computing has a superior advantage in tackling specific problems, such as integer factorization and Simon's problem. For more general tasks in machine learning, by applying variational quantum circuits, more and more quantum algorithms have been proposed recently, especially in supervised learning and unsupervised learning. However, little work has been done in reinforcement learning, arguably more important and challenging. Previous work in quantum reinforcement learning mainly focuses on discrete control tasks where the action space is discrete. In this work, we develop a quantum reinforcement learning algorithm based on soft actor-critic -- one of the state-of-the-art methods for continuous control. Specifically, we use a hybrid quantum-classical policy network consisting of a variational quantum circuit and a classical artificial neural network. Tested in a standard reinforcement learning benchmark, we show that this quantum version of soft actor-critic is comparable with the original soft actor-critic, using much less adjustable parameters. Furthermore, we analyze the effect of different hyper-parameters and policy network architectures, pointing out the importance of architecture design for quantum reinforcement learning.

Related papers

• Robustness and Generalization in Quantum Reinforcement Learning via Lipschitz Regularization [2.8445375187526154]
  We propose a regularized version of a quantum policy gradient approach, named the RegQPG algorithm. We show that training with RegQPG improves the robustness and generalization of the resulting policies.
  arXiv  Detail & Related papers  (2024-10-28T15:20:35Z)
• Hybrid Quantum-Classical Reinforcement Learning in Latent Observation Spaces [0.36944296923226316]
  Recent progress in quantum machine learning has sparked interest in using quantum methods to tackle classical control problems. We propose to solve this dimensionality challenge by a classical autoencoder and a quantum agent together. A series of numerical experiments are designed for a performance analysis of the latent-space learning method.
  arXiv  Detail & Related papers  (2024-10-23T21:19:38Z)
• Entanglement-Assisted Quantum Networks: Mechanics, Enabling Technologies, Challenges, and Research Directions [66.27337498864556]
  This paper presents a comprehensive survey of entanglement-assisted quantum networks. It provides a detailed overview of the network structure, working principles, and development stages. It also emphasizes open research directions, including architecture design, entanglement-based network issues, and standardization.
  arXiv  Detail & Related papers  (2023-07-24T02:48:22Z)
• QNEAT: Natural Evolution of Variational Quantum Circuit Architecture [95.29334926638462]
  We focus on variational quantum circuits (VQC), which emerged as the most promising candidates for the quantum counterpart of neural networks. Although showing promising results, VQCs can be hard to train because of different issues, e.g., barren plateau, periodicity of the weights, or choice of architecture. We propose a gradient-free algorithm inspired by natural evolution to optimize both the weights and the architecture of the VQC.
  arXiv  Detail & Related papers  (2023-04-14T08:03:20Z)
• Quantum Deep Hedging [10.243020478772056]
  We look at the problem of hedging where deep reinforcement learning offers a powerful framework for real markets. We develop quantum reinforcement learning methods based on policy-search and distributional actor-critic algorithms. We successfully implement the proposed models on a trapped-ion quantum processor.
  arXiv  Detail & Related papers  (2023-03-29T10:42:50Z)
• Quantum Machine Learning: from physics to software engineering [58.720142291102135]
  We show how classical machine learning approach can help improve the facilities of quantum computers. We discuss how quantum algorithms and quantum computers may be useful for solving classical machine learning tasks.
  arXiv  Detail & Related papers  (2023-01-04T23:37:45Z)
• The Quantum Path Kernel: a Generalized Quantum Neural Tangent Kernel for Deep Quantum Machine Learning [52.77024349608834]
  Building a quantum analog of classical deep neural networks represents a fundamental challenge in quantum computing. Key issue is how to address the inherent non-linearity of classical deep learning. We introduce the Quantum Path Kernel, a formulation of quantum machine learning capable of replicating those aspects of deep machine learning.
  arXiv  Detail & Related papers  (2022-12-22T16:06:24Z)
• Optimizing Tensor Network Contraction Using Reinforcement Learning [86.05566365115729]
  We propose a Reinforcement Learning (RL) approach combined with Graph Neural Networks (GNN) to address the contraction ordering problem. The problem is extremely challenging due to the huge search space, the heavy-tailed reward distribution, and the challenging credit assignment. We show how a carefully implemented RL-agent that uses a GNN as the basic policy construct can address these challenges.
  arXiv  Detail & Related papers  (2022-04-18T21:45:13Z)
• Quantum Deep Reinforcement Learning for Robot Navigation Tasks [2.6999000177990924]
  We show that quantum circuits in hybrid quantum-classic reinforcement learning setups are capable of learning optimal policies in multiple robotic navigation scenarios. We find that the employed quantum circuits outperform the classical neural network baselines when equating for the number of trainable parameters.
  arXiv  Detail & Related papers  (2022-02-24T16:26:23Z)
• Quantum algorithms for quantum dynamics: A performance study on the spin-boson model [68.8204255655161]
  Quantum algorithms for quantum dynamics simulations are traditionally based on implementing a Trotter-approximation of the time-evolution operator. variational quantum algorithms have become an indispensable alternative, enabling small-scale simulations on present-day hardware. We show that, despite providing a clear reduction of quantum gate cost, the variational method in its current implementation is unlikely to lead to a quantum advantage.
  arXiv  Detail & Related papers  (2021-08-09T18:00:05Z)
• Variational quantum policies for reinforcement learning [0.0]
  Variational quantum circuits have recently gained popularity as quantum machine learning models. In this work, we investigate how to construct and train reinforcement learning policies based on variational quantum circuits.
  arXiv  Detail & Related papers  (2021-03-09T17:33:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.