Benchmarking Quantum Reinforcement Learning

• URL: http://arxiv.org/abs/2502.04909v1
• Date: Fri, 07 Feb 2025 13:28:20 GMT
• Title: Benchmarking Quantum Reinforcement Learning
• Authors: Georg Kruse, Rodrigo Coelho, Andreas Rosskopf, Robert Wille, Jeanette-Miriam Lorenz,
• Abstract summary: Quantum Reinforcement Learning (QRL) has emerged as a promising research field, leveraging the principles of quantum mechanics to enhance the performance of reinforcement learning (RL) algorithms. It is still uncertain if QRL can show any advantage over classical RL beyond artificial problem formulations. It is not yet clear which streams of QRL research show the greatest potential.
• Score: 2.536162003546062
• License:
• Abstract: Quantum Reinforcement Learning (QRL) has emerged as a promising research field, leveraging the principles of quantum mechanics to enhance the performance of reinforcement learning (RL) algorithms. However, despite its growing interest, QRL still faces significant challenges. It is still uncertain if QRL can show any advantage over classical RL beyond artificial problem formulations. Additionally, it is not yet clear which streams of QRL research show the greatest potential. The lack of a unified benchmark and the need to evaluate the reliance on quantum principles of QRL approaches are pressing questions. This work aims to address these challenges by providing a comprehensive comparison of three major QRL classes: Parameterized Quantum Circuit based QRL (PQC-QRL) (with one policy gradient (QPG) and one Q-Learning (QDQN) algorithm), Free Energy based QRL (FE-QRL), and Amplitude Amplification based QRL (AA-QRL). We introduce a set of metrics to evaluate the QRL algorithms on the widely applicable benchmark of gridworld games. Our results provide a detailed analysis of the strengths and weaknesses of the QRL classes, shedding light on the role of quantum principles in QRL and paving the way for future research in this field.

Related papers

• SF-DQN: Provable Knowledge Transfer using Successor Feature for Deep Reinforcement Learning [89.04776523010409]
  This paper studies the transfer reinforcement learning (RL) problem where multiple RL problems have different reward functions but share the same underlying transition dynamics. In this setting, the Q-function of each RL problem (task) can be decomposed into a successor feature (SF) and a reward mapping. We establish the first convergence analysis with provable generalization guarantees for SF-DQN with GPI.
  arXiv  Detail & Related papers  (2024-05-24T20:30:14Z)
• Efficient quantum recurrent reinforcement learning via quantum reservoir computing [3.6881738506505988]
  Quantum reinforcement learning (QRL) has emerged as a framework to solve sequential decision-making tasks. This work presents a novel approach to address this challenge by constructing QRL agents utilizing QRNN-based quantum long short-term memory (QLSTM)
  arXiv  Detail & Related papers  (2023-09-13T22:18:38Z)
• Quantum Imitation Learning [74.15588381240795]
  We propose quantum imitation learning (QIL) with a hope to utilize quantum advantage to speed up IL. We develop two QIL algorithms, quantum behavioural cloning (Q-BC) and quantum generative adversarial imitation learning (Q-GAIL) Experiment results demonstrate that both Q-BC and Q-GAIL can achieve comparable performance compared to classical counterparts.
  arXiv  Detail & Related papers  (2023-04-04T12:47:35Z)
• Optimal Goal-Reaching Reinforcement Learning via Quasimetric Learning [73.80728148866906]
  Quasimetric Reinforcement Learning (QRL) is a new RL method that utilizes quasimetric models to learn optimal value functions. On offline and online goal-reaching benchmarks, QRL also demonstrates improved sample efficiency and performance.
  arXiv  Detail & Related papers  (2023-04-03T17:59:58Z)
• Asynchronous training of quantum reinforcement learning [0.8702432681310399]
  A leading method of building quantum RL agents relies on the variational quantum circuits (VQCs) In this paper, we approach this challenge through asynchronous training QRL agents. We demonstrate the results via numerical simulations that within the tasks considered, the asynchronous training of QRL agents can reach performance comparable to or superior.
  arXiv  Detail & Related papers  (2023-01-12T15:54:44Z)
• Quantum deep recurrent reinforcement learning [0.8702432681310399]
  Reinforcement learning (RL) is one of the machine learning (ML) paradigms which can be used to solve complex sequential decision making problems. We build a quantum long short-term memory (QLSTM) to be the core of the QRL agent and train the whole model with deep $Q$-learning. We demonstrate the results via numerical simulations that the QLSTM-DRQN can solve standard benchmark such as Cart-Pole with more stable and higher average scores than classical DRQN.
  arXiv  Detail & Related papers  (2022-10-26T17:29:19Z)
• LCRL: Certified Policy Synthesis via Logically-Constrained Reinforcement Learning [78.2286146954051]
  LCRL implements model-free Reinforcement Learning (RL) algorithms over unknown Decision Processes (MDPs) We present case studies to demonstrate the applicability, ease of use, scalability, and performance of LCRL.
  arXiv  Detail & Related papers  (2022-09-21T13:21:00Z)
• Optimizing Tensor Network Contraction Using Reinforcement Learning [86.05566365115729]
  We propose a Reinforcement Learning (RL) approach combined with Graph Neural Networks (GNN) to address the contraction ordering problem. The problem is extremely challenging due to the huge search space, the heavy-tailed reward distribution, and the challenging credit assignment. We show how a carefully implemented RL-agent that uses a GNN as the basic policy construct can address these challenges.
  arXiv  Detail & Related papers  (2022-04-18T21:45:13Z)
• Quantum Multi-Agent Reinforcement Learning via Variational Quantum Circuit Design [16.53719091025918]
  This paper extends and demonstrates the QRL to quantum multi-agent RL (QMARL) The extension of QRL to QMARL is not straightforward due to the challenge of the noise intermediate-scale quantum (NISQ) and the non-stationary properties in classical multi-agent RL (MARL) The proposed QMARL framework enhances 57.7% of total reward than classical frameworks.
  arXiv  Detail & Related papers  (2022-03-20T03:44:45Z)
• Quantum agents in the Gym: a variational quantum algorithm for deep Q-learning [0.0]
  We introduce a training method for parametrized quantum circuits (PQCs) that can be used to solve RL tasks for discrete and continuous state spaces. We investigate which architectural choices for quantum Q-learning agents are most important for successfully solving certain types of environments.
  arXiv  Detail & Related papers  (2021-03-28T08:57:22Z)
• EMaQ: Expected-Max Q-Learning Operator for Simple Yet Effective Offline and Online RL [48.552287941528]
  Off-policy reinforcement learning holds the promise of sample-efficient learning of decision-making policies. In the offline RL setting, standard off-policy RL methods can significantly underperform. We introduce Expected-Max Q-Learning (EMaQ), which is more closely related to the resulting practical algorithm.
  arXiv  Detail & Related papers  (2020-07-21T21:13:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.