Variational Quantum Circuit Design for Quantum Reinforcement Learning on Continuous Environments

• URL: http://arxiv.org/abs/2312.13798v1
• Date: Thu, 21 Dec 2023 12:40:01 GMT
• Title: Variational Quantum Circuit Design for Quantum Reinforcement Learning on Continuous Environments
• Abstract summary: Quantum Reinforcement Learning (QRL) emerged as a branch of reinforcement learning (RL) that uses quantum submodules in the architecture of the algorithm. One branch of QRL focuses on the replacement of neural networks (NN) by variational quantum circuits (VQC) as function approximators. We show how to design a QRL agent in order to solve classical environments with continuous action spaces and benchmark our agents against classical feed-forward NNs.
• Score: 2.9723999564214267
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Quantum Reinforcement Learning (QRL) emerged as a branch of reinforcement learning (RL) that uses quantum submodules in the architecture of the algorithm. One branch of QRL focuses on the replacement of neural networks (NN) by variational quantum circuits (VQC) as function approximators. Initial works have shown promising results on classical environments with discrete action spaces, but many of the proposed architectural design choices of the VQC lack a detailed investigation. Hence, in this work we investigate the impact of VQC design choices such as angle embedding, encoding block architecture and postprocessesing on the training capabilities of QRL agents. We show that VQC design greatly influences training performance and heuristically derive enhancements for the analyzed components. Additionally, we show how to design a QRL agent in order to solve classical environments with continuous action spaces and benchmark our agents against classical feed-forward NNs.

Related papers

• Quantum LEGO Learning: A Modular Design Principle for Hybrid Artificial Intelligence [63.39968536637762]
  We introduce Quantum LEGO Learning, a learning framework that treats classical and quantum components as reusable, composable learning blocks.<n>Within this framework, a pre-trained classical neural network serves as a frozen feature block, while a VQC acts as a trainable adaptive module.<n>We develop a block-wise generalization theory that decomposes learning error into approximation and estimation components.
  arXiv  Detail & Related papers  (2026-01-29T14:29:21Z)
• Variational Quantum Circuit-Based Reinforcement Learning for Dynamic Portfolio Optimization [7.349651640835185]
  This paper presents a Quantum Reinforcement Learning solution to the dynamic portfolio optimization problem based on Variational Quantum Circuits.<n>We show that our quantum agents achieve risk-adjusted performance comparable to, and in some cases exceeding, that of classical Deep RL models.
  arXiv  Detail & Related papers  (2026-01-20T15:17:24Z)
• Quantum Visual Fields with Neural Amplitude Encoding [70.86293548779774]
  We introduce a new type of Quantum Implicit Neural Representation (QINR) for 2D image and 3D geometric field learning.<n>QVF encodes classical data into quantum statevectors using neural amplitude encoding grounded in a learnable energy manifold.<n>Our ansatz follows a fully entangled design of learnable parametrised quantum circuits, with quantum (unitary) operations performed in the real Hilbert space.
  arXiv  Detail & Related papers  (2025-08-14T17:59:52Z)
• Quantum Circuit Structure Optimization for Quantum Reinforcement Learning [9.913187216180424]
  Reinforcement learning enables agents to learn optimal policies through environmental interaction.<n>Quantum reinforcement learning (QRL) addresses this issue by leveraging superposition and entanglement in quantum computing.<n>This paper proposes a QRL-NAS algorithm that integrates quantum neural architecture search (QNAS) to optimize PQC structures within QRL.
  arXiv  Detail & Related papers  (2025-07-01T09:16:58Z)
• VQC-MLPNet: An Unconventional Hybrid Quantum-Classical Architecture for Scalable and Robust Quantum Machine Learning [50.95799256262098]
  Variational quantum circuits (VQCs) hold promise for quantum machine learning but face challenges in expressivity, trainability, and noise resilience.<n>We propose VQC-MLPNet, a hybrid architecture where a VQC generates the first-layer weights of a classical multilayer perceptron during training, while inference is performed entirely classically.
  arXiv  Detail & Related papers  (2025-06-12T01:38:15Z)
• RhoDARTS: Differentiable Quantum Architecture Search with Density Matrix Simulations [48.670876200492415]
  Variational Quantum Algorithms (VQAs) are a promising approach for leveraging powerful Noisy Intermediate-Scale Quantum (NISQ) computers.<n>We propose $rho$DARTS, a differentiable Quantum Architecture Search (QAS) algorithm that models the search process as the evolution of a quantum mixed state.
  arXiv  Detail & Related papers  (2025-06-04T08:30:35Z)
• Differentiable Quantum Architecture Search in Asynchronous Quantum Reinforcement Learning [3.6881738506505988]
  We propose differentiable quantum architecture search (DiffQAS) to enable trainable circuit parameters and structure weights. We show that our proposed DiffQAS-QRL approach achieves performance comparable to manually-crafted circuit architectures.
  arXiv  Detail & Related papers  (2024-07-25T17:11:00Z)
• Weight Re-Mapping for Variational Quantum Algorithms [54.854986762287126]
  We introduce the concept of weight re-mapping for variational quantum circuits (VQCs) We employ seven distinct weight re-mapping functions to assess their impact on eight classification datasets. Our results indicate that weight re-mapping can enhance the convergence speed of the VQC.
  arXiv  Detail & Related papers  (2023-06-09T09:42:21Z)
• QNEAT: Natural Evolution of Variational Quantum Circuit Architecture [95.29334926638462]
  We focus on variational quantum circuits (VQC), which emerged as the most promising candidates for the quantum counterpart of neural networks. Although showing promising results, VQCs can be hard to train because of different issues, e.g., barren plateau, periodicity of the weights, or choice of architecture. We propose a gradient-free algorithm inspired by natural evolution to optimize both the weights and the architecture of the VQC.
  arXiv  Detail & Related papers  (2023-04-14T08:03:20Z)
• Quantum Imitation Learning [74.15588381240795]
  We propose quantum imitation learning (QIL) with a hope to utilize quantum advantage to speed up IL. We develop two QIL algorithms, quantum behavioural cloning (Q-BC) and quantum generative adversarial imitation learning (Q-GAIL) Experiment results demonstrate that both Q-BC and Q-GAIL can achieve comparable performance compared to classical counterparts.
  arXiv  Detail & Related papers  (2023-04-04T12:47:35Z)
• A Quantum Kernel Learning Approach to Acoustic Modeling for Spoken Command Recognition [69.97260364850001]
  We propose a quantum kernel learning (QKL) framework to address the inherent data sparsity issues. We project acoustic features based on classical-to-quantum feature encoding.
  arXiv  Detail & Related papers  (2022-11-02T16:46:23Z)
• Learning capability of parametrized quantum circuits [2.51657752676152]
  Variational quantum algorithms (VQAs) and their applications in the field of quantum machine learning through parametrized quantum circuits (PQCs) are thought to be one major way of leveraging noisy intermediate-scale quantum computing devices. In this paper, we build upon the work by Schuld et al. and compare popular ans"atze for PQCs through the new measure of learning capability. We also examine dissipative quantum neural networks (dQNN) as introduced by Beer et al. and propose a data re-upload structure for dQNNs to increase their learning capability.
  arXiv  Detail & Related papers  (2022-09-21T13:26:20Z)
• Evolutionary Quantum Architecture Search for Parametrized Quantum Circuits [7.298440208725654]
  We introduce EQAS-PQC, an evolutionary quantum architecture search framework for PQC-based models. We show that our method can significantly improve the performance of hybrid quantum-classical models.
  arXiv  Detail & Related papers  (2022-08-23T19:47:37Z)
• Optimizing Tensor Network Contraction Using Reinforcement Learning [86.05566365115729]
  We propose a Reinforcement Learning (RL) approach combined with Graph Neural Networks (GNN) to address the contraction ordering problem. The problem is extremely challenging due to the huge search space, the heavy-tailed reward distribution, and the challenging credit assignment. We show how a carefully implemented RL-agent that uses a GNN as the basic policy construct can address these challenges.
  arXiv  Detail & Related papers  (2022-04-18T21:45:13Z)
• On Circuit-based Hybrid Quantum Neural Networks for Remote Sensing Imagery Classification [88.31717434938338]
  The hybrid QCNNs enrich the classical architecture of CNNs by introducing a quantum layer within a standard neural network. The novel QCNN proposed in this work is applied to the Land Use and Land Cover (LULC) classification, chosen as an Earth Observation (EO) use case. The results of the multiclass classification prove the effectiveness of the presented approach, by demonstrating that the QCNN performances are higher than the classical counterparts.
  arXiv  Detail & Related papers  (2021-09-20T12:41:50Z)
• Quantum agents in the Gym: a variational quantum algorithm for deep Q-learning [0.0]
  We introduce a training method for parametrized quantum circuits (PQCs) that can be used to solve RL tasks for discrete and continuous state spaces. We investigate which architectural choices for quantum Q-learning agents are most important for successfully solving certain types of environments.
  arXiv  Detail & Related papers  (2021-03-28T08:57:22Z)
• Neural Predictor based Quantum Architecture Search [15.045985536395479]
  Variational quantum algorithms (VQAs) are widely speculated to deliver quantum advantages for practical problems under the quantum-classical hybrid computational paradigm in the near term. In this work, we propose to use a neural network based predictor as the evaluation policy for quantum architecture search (QAS)
  arXiv  Detail & Related papers  (2021-03-11T08:26:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.