Multi-Agent Quantum Reinforcement Learning using Evolutionary Optimization

• URL: http://arxiv.org/abs/2311.05546v4
• Date: Thu, 02 Jan 2025 14:09:53 GMT
• Title: Multi-Agent Quantum Reinforcement Learning using Evolutionary Optimization
• Authors: Michael Kölle, Felix Topp, Thomy Phan, Philipp Altmann, Jonas Nüßlein, Claudia Linnhoff-Popien,
• Abstract summary: We build upon an existing approach for gradient free Quantum Reinforcement Learning and propose three genetic variations with Variational Quantum Circuits for Multi-Agent Reinforcement Learning. We show that our Variational Quantum Circuit approaches perform significantly better compared to a neural network with a similar amount of trainable parameters.
• Score: 6.3443033704469
• License:
• Abstract: Multi-Agent Reinforcement Learning is becoming increasingly more important in times of autonomous driving and other smart industrial applications. Simultaneously a promising new approach to Reinforcement Learning arises using the inherent properties of quantum mechanics, reducing the trainable parameters of a model significantly. However, gradient-based Multi-Agent Quantum Reinforcement Learning methods often have to struggle with barren plateaus, holding them back from matching the performance of classical approaches. While gradient free Quantum Reinforcement Learning methods may alleviate some of these challenges, they too are not immune to the difficulties posed by barren plateaus. We build upon an existing approach for gradient free Quantum Reinforcement Learning and propose three genetic variations with Variational Quantum Circuits for Multi-Agent Reinforcement Learning using evolutionary optimization. We evaluate our genetic variations in the Coin Game environment and also compare them to classical approaches. We showed that our Variational Quantum Circuit approaches perform significantly better compared to a neural network with a similar amount of trainable parameters. Compared to the larger neural network, our approaches archive similar results using $97.88\%$ less parameters.

Related papers

• Leveraging Pre-Trained Neural Networks to Enhance Machine Learning with Variational Quantum Circuits [48.33631905972908]
  We introduce an innovative approach that utilizes pre-trained neural networks to enhance Variational Quantum Circuits (VQC) This technique effectively separates approximation error from qubit count and removes the need for restrictive conditions. Our results extend to applications such as human genome analysis, demonstrating the broad applicability of our approach.
  arXiv  Detail & Related papers  (2024-11-13T12:03:39Z)
• Architectural Influence on Variational Quantum Circuits in Multi-Agent Reinforcement Learning: Evolutionary Strategies for Optimization [5.7918970571149835]
  We propose a Gate-Based, a Layer-Based, and a Prototype-Based concept to mutate and recombine quantum circuits. Our results show the best performance for mutation-only strategies and the Gate-Based approach. In particular, we observe a significantly better score, higher total and own collected coins, as well as a superior own coin rate for the best agent when evaluated in the Coin Game environment.
  arXiv  Detail & Related papers  (2024-07-30T11:16:25Z)
• Evolutionary-enhanced quantum supervised learning model [0.0]
  This study proposes an evolutionary-enhanced ansatz-free supervised learning model. In contrast to parametrized circuits, our model employs circuits with variable topology that evolves through an elitist method. Our framework successfully avoids barren plateaus, resulting in enhanced model accuracy.
  arXiv  Detail & Related papers  (2023-11-14T11:08:47Z)
• QNEAT: Natural Evolution of Variational Quantum Circuit Architecture [95.29334926638462]
  We focus on variational quantum circuits (VQC), which emerged as the most promising candidates for the quantum counterpart of neural networks. Although showing promising results, VQCs can be hard to train because of different issues, e.g., barren plateau, periodicity of the weights, or choice of architecture. We propose a gradient-free algorithm inspired by natural evolution to optimize both the weights and the architecture of the VQC.
  arXiv  Detail & Related papers  (2023-04-14T08:03:20Z)
• Quantum Deep Hedging [10.243020478772056]
  We look at the problem of hedging where deep reinforcement learning offers a powerful framework for real markets. We develop quantum reinforcement learning methods based on policy-search and distributional actor-critic algorithms. We successfully implement the proposed models on a trapped-ion quantum processor.
  arXiv  Detail & Related papers  (2023-03-29T10:42:50Z)
• The Quantum Path Kernel: a Generalized Quantum Neural Tangent Kernel for Deep Quantum Machine Learning [52.77024349608834]
  Building a quantum analog of classical deep neural networks represents a fundamental challenge in quantum computing. Key issue is how to address the inherent non-linearity of classical deep learning. We introduce the Quantum Path Kernel, a formulation of quantum machine learning capable of replicating those aspects of deep machine learning.
  arXiv  Detail & Related papers  (2022-12-22T16:06:24Z)
• Towards Multi-Agent Reinforcement Learning using Quantum Boltzmann Machines [2.015864965523243]
  We propose an extension to the original concept in order to solve more challenging problems. We add an experience replay buffer and use different networks for approximating the target and policy values. Quantum sampling proves to be a promising method for reinforcement learning tasks, but is currently limited by the QPU size.
  arXiv  Detail & Related papers  (2021-09-22T17:59:24Z)
• Optimizing Quantum Variational Circuits with Deep Reinforcement Learning [0.0]
  We evaluate the potential of contemporary methods in deep reinforcement learning to augment gradient based optimization routines in quantum variational circuits. We find that reinforcement learning augmenteds consistently outperform gradient descent in noisy environments.
  arXiv  Detail & Related papers  (2021-09-07T16:48:39Z)
• Efficient Model-Based Multi-Agent Mean-Field Reinforcement Learning [89.31889875864599]
  We propose an efficient model-based reinforcement learning algorithm for learning in multi-agent systems. Our main theoretical contributions are the first general regret bounds for model-based reinforcement learning for MFC. We provide a practical parametrization of the core optimization problem.
  arXiv  Detail & Related papers  (2021-07-08T18:01:02Z)
• Variational Quantum Optimization with Multi-Basis Encodings [62.72309460291971]
  We introduce a new variational quantum algorithm that benefits from two innovations: multi-basis graph complexity and nonlinear activation functions. Our results in increased optimization performance, two increase in effective landscapes and a reduction in measurement progress.
  arXiv  Detail & Related papers  (2021-06-24T20:16:02Z)
• Gone Fishing: Neural Active Learning with Fisher Embeddings [55.08537975896764]
  There is an increasing need for active learning algorithms that are compatible with deep neural networks. This article introduces BAIT, a practical representation of tractable, and high-performing active learning algorithm for neural networks.
  arXiv  Detail & Related papers  (2021-06-17T17:26:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.