Quantum Hierarchical Reinforcement Learning via Variational Quantum Circuits

• URL: http://arxiv.org/abs/2605.03434v1
• Date: Tue, 05 May 2026 07:14:53 GMT
• Title: Quantum Hierarchical Reinforcement Learning via Variational Quantum Circuits
• Abstract summary: We develop a hybrid hierarchical agent based on the option-critic architecture.<n>This hybrid agent substitutes classical components with variational quantum circuits for feature extractors, option-value functions, termination functions, and intra-option policies.
• Score: 14.345235373434242
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reinforcement learning is one of the most challenging learning paradigms where efficacy and efficiency gains are extremely valuable. Hierarchical reinforcement learning is a variant that leverages temporal abstraction to structure decision-making. While parametrized quantum computations have shown success in non-hierarchical reinforcement learning, whether these advantages adapt to hierarchical decision-making remains a critical open question. In this work, we develop a hybrid hierarchical agent based on the option-critic architecture. This hybrid agent substitutes classical components with variational quantum circuits for feature extractors, option-value functions, termination functions, and intra-option policies. Evaluated on standard benchmarking environments, results show that a hybrid agent utilizing a quantum feature extractor can outperform classical baselines while saving up to 66\% trainable parameters. We also identify an architectural bottleneck that quantum option-value estimation severely degrades performance. Further ablation studies reveal how architectural choices of the quantum circuits affect performance. Our work establishes design principles for parameter-efficient hybrid hierarchical agents.

Related papers

• Quantum LEGO Learning: A Modular Design Principle for Hybrid Artificial Intelligence [63.39968536637762]
  We introduce Quantum LEGO Learning, a learning framework that treats classical and quantum components as reusable, composable learning blocks.<n>Within this framework, a pre-trained classical neural network serves as a frozen feature block, while a VQC acts as a trainable adaptive module.<n>We develop a block-wise generalization theory that decomposes learning error into approximation and estimation components.
  arXiv  Detail & Related papers  (2026-01-29T14:29:21Z)
• Regression of Functions by Quantum Neural Networks Circuits [41.09105068326236]
  This work investigates automated quantum-circuit construction for regression tasks.<n>It introduces a genetic-algorithm framework that discovers Reduced Regressor QNN architectures.
  arXiv  Detail & Related papers  (2025-12-23T01:58:03Z)
• Hybrid action Reinforcement Learning for quantum architecture search [22.797945771013037]
  HyRLQAS is a unified framework that integrates discrete gate placement and continuous parameter generation.<n>We show that HyRLQAS consistently achieves lower energy errors and more compact circuit structures.<n>These findings suggest that hybrid-action reinforcement learning offers a principled pathway toward automated and hardware-efficient quantum circuit design.
  arXiv  Detail & Related papers  (2025-11-07T04:00:24Z)
• Reinforcement Learning for Quantum Network Control with Application-Driven Objectives [53.03367590211247]
  Dynamic programming and reinforcement learning offer promising tools for optimizing control strategies.<n>We propose a novel RL framework that directly optimize non-linear, differentiable objective functions.<n>Our work comprises the first step towards non-linear objective function optimization in quantum networks with RL, opening a path towards more advanced use cases.
  arXiv  Detail & Related papers  (2025-09-12T18:41:10Z)
• On the Importance of Fundamental Properties in Quantum-Classical Machine Learning Models [0.0]
  We present a systematic study of how quantum circuit design affects the performance of hybrid quantum-classical neural networks on a causal classification task.<n>The architecture combines a convolutional neural network for classical feature extraction with a parameterized quantum circuit acting as the quantum layer.<n>Results show that increasing the number of ansatz repetitions improves generalization and training stability, though benefits tend to plateau beyond a certain depth.<n>The choice of feature mapping is even more critical: only encodings with multi-axis Pauli rotations enable successful learning, while simpler maps lead to underfitting or loss of class separability.
  arXiv  Detail & Related papers  (2025-07-14T11:18:53Z)
• Provably Robust Training of Quantum Circuit Classifiers Against Parameter Noise [49.97673761305336]
  Noise remains a major obstacle to achieving reliable quantum algorithms.<n>We present a provably noise-resilient training theory and algorithm to enhance the robustness of parameterized quantum circuit classifiers.
  arXiv  Detail & Related papers  (2025-05-24T02:51:34Z)
• Untrained Filtering with Trained Focusing for Superior Quantum Architecture Search [14.288836269941207]
  Quantum architecture search (QAS) represents a fundamental challenge in quantum machine learning. We decompose the search process into dynamic alternating phases of coarse and fine-grained knowledge learning. QUEST-A develops an evolutionary mechanism with knowledge accumulation and reuse to enhance multi-level knowledge transfer.
  arXiv  Detail & Related papers  (2024-10-31T01:57:14Z)
• Bayesian Parameterized Quantum Circuit Optimization (BPQCO): A task and hardware-dependent approach [49.89480853499917]
  Variational quantum algorithms (VQA) have emerged as a promising quantum alternative for solving optimization and machine learning problems. In this paper, we experimentally demonstrate the influence of the circuit design on the performance obtained for two classification problems. We also study the degradation of the obtained circuits in the presence of noise when simulating real quantum computers.
  arXiv  Detail & Related papers  (2024-04-17T11:00:12Z)
• Quantum Advantage Actor-Critic for Reinforcement Learning [5.579028648465784]
  We propose a novel quantum reinforcement learning approach that combines the Advantage Actor-Critic algorithm with variational quantum circuits. We empirically test multiple quantum Advantage Actor-Critic configurations with the well known Cart Pole environment to evaluate our approach in control tasks with continuous state spaces.
  arXiv  Detail & Related papers  (2024-01-13T11:08:45Z)
• Pre-training Tensor-Train Networks Facilitates Machine Learning with Variational Quantum Circuits [70.97518416003358]
  Variational quantum circuits (VQCs) hold promise for quantum machine learning on noisy intermediate-scale quantum (NISQ) devices. While tensor-train networks (TTNs) can enhance VQC representation and generalization, the resulting hybrid model, TTN-VQC, faces optimization challenges due to the Polyak-Lojasiewicz (PL) condition. To mitigate this challenge, we introduce Pre+TTN-VQC, a pre-trained TTN model combined with a VQC.
  arXiv  Detail & Related papers  (2023-05-18T03:08:18Z)
• Quantum Machine Learning with SQUID [64.53556573827525]
  We present the Scaled QUantum IDentifier (SQUID), an open-source framework for exploring hybrid Quantum-Classical algorithms for classification problems. We provide examples of using SQUID in a standard binary classification problem from the popular MNIST dataset.
  arXiv  Detail & Related papers  (2021-04-30T21:34:11Z)
• Quantum circuit architecture search for variational quantum algorithms [88.71725630554758]
  We propose a resource and runtime efficient scheme termed quantum architecture search (QAS) QAS automatically seeks a near-optimal ansatz to balance benefits and side-effects brought by adding more noisy quantum gates. We implement QAS on both the numerical simulator and real quantum hardware, via the IBM cloud, to accomplish data classification and quantum chemistry tasks.
  arXiv  Detail & Related papers  (2020-10-20T12:06:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.