On Quantum Natural Policy Gradients

• URL: http://arxiv.org/abs/2401.08307v1
• Date: Tue, 16 Jan 2024 12:08:31 GMT
• Title: On Quantum Natural Policy Gradients
• Abstract summary: This research delves into the role of the quantum Fisher Information Matrix (FIM) in enhancing the performance of reinforcement learning agents. Our results indicate that a PQC-based agent using the quantum FIM without additional insights typically incurs a larger approximation error and does not guarantee improved performance compared to the classical FIM.
• Score: 0.0
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: This research delves into the role of the quantum Fisher Information Matrix (FIM) in enhancing the performance of Parameterized Quantum Circuit (PQC)-based reinforcement learning agents. While previous studies have highlighted the effectiveness of PQC-based policies preconditioned with the quantum FIM in contextual bandits, its impact in broader reinforcement learning contexts, such as Markov Decision Processes, is less clear. Through a detailed analysis of L\"owner inequalities between quantum and classical FIMs, this study uncovers the nuanced distinctions and implications of using each type of FIM. Our results indicate that a PQC-based agent using the quantum FIM without additional insights typically incurs a larger approximation error and does not guarantee improved performance compared to the classical FIM. Empirical evaluations in classic control benchmarks suggest even though quantum FIM preconditioning outperforms standard gradient ascent, in general it is not superior to classical FIM preconditioning.

Related papers

• In-Context Learning as Implicit Policy Gradient [47.12608115550359]
  Large language models can iteratively improve their outputs by incorporating generated samples and their corresponding evaluation scores as in-context examples.<n>We show that score-conditioned In-Context Learning (ICL) admits a structural correspondence to policy gradient optimization.
  arXiv  Detail & Related papers  (2026-07-25T11:13:52Z)
• Error-mitigation aware benchmarking strategy for quantum optimization problems [3.026585988755882]
  entropy benchmarking does not account for finite-shot effects or for quantum error mitigation.<n>We develop a benchmarking framework that explicitly incorporates finite-shot statistics and the resource overhead induced by QEM.<n>Our framework quantifies quantum advantage through the confidence that an estimated energy lies within an interval defined by the best-known classical upper and lower bounds.
  arXiv  Detail & Related papers  (2026-01-26T16:55:47Z)
• Continual Quantum Architecture Search with Tensor-Train Encoding: Theory and Applications to Signal Processing [68.35481158940401]
  CL-QAS is a continual quantum architecture search framework.<n>It mitigates challenges of costly encoding amplitude and forgetting in variational quantum circuits.<n>It achieves controllable robustness expressivity, sample-efficient generalization, and smooth convergence without barren plateaus.
  arXiv  Detail & Related papers  (2026-01-10T02:36:03Z)
• Dissecting Quantum Reinforcement Learning: A Systematic Evaluation of Key Components [0.08921166277011346]
  Quantum Reinforcement Learning (QRL) has emerged as a promising paradigm at the intersection of quantum computing and reinforcement learning.<n>By design, PQCs create hybrid quantum-classical models, but their practical applicability remains uncertain.
  arXiv  Detail & Related papers  (2025-11-21T10:21:39Z)
• Hybrid Quantum-Classical Policy Gradient for Adaptive Control of Cyber-Physical Systems: A Comparative Study of VQC vs. MLP [0.0]
  The study employed a multilayer perceptron (MLP) agent as a classical baseline and a parameterized variational quantum circuit (VQC) as a quantum counterpart.<n> Empirical results demonstrated that the classical achieved near-optimal policy convergence with a mean return of 498.7 +/- 3.2.<n>The VQC exhibited limited learning capability, with an average return of 14.6 +/- 4.8, primarily constrained by circuit depth and qubit connectivity.
  arXiv  Detail & Related papers  (2025-10-07T15:09:29Z)
• Conditional Advantage Estimation for Reinforcement Learning in Large Reasoning Models [50.84995206660551]
  We introduce Conditional advANtage estimatiON (CANON) to amplify the impact of a target metric without presuming its direction.<n>CANON based on entropy consistently outperforms prior methods on both math reasoning and high-complexity logic tasks.
  arXiv  Detail & Related papers  (2025-09-28T16:33:07Z)
• Quantum Reinforcement Learning-Guided Diffusion Model for Image Synthesis via Hybrid Quantum-Classical Generative Model Architectures [2.005299372367689]
  We introduce a quantum reinforcement learning (QRL) controller that dynamically adjusts CFG at each denoising step.<n>The controller adopts a hybrid quantum--classical actor--critic architecture.<n> Experiments on CIFAR-10 demonstrate that our QRL policy improves perceptual quality.
  arXiv  Detail & Related papers  (2025-09-17T16:47:04Z)
• Reinforcement Learning for Quantum Network Control with Application-Driven Objectives [53.03367590211247]
  Dynamic programming and reinforcement learning offer promising tools for optimizing control strategies.<n>We propose a novel RL framework that directly optimize non-linear, differentiable objective functions.<n>Our work comprises the first step towards non-linear objective function optimization in quantum networks with RL, opening a path towards more advanced use cases.
  arXiv  Detail & Related papers  (2025-09-12T18:41:10Z)
• TensoMeta-VQC: A Tensor-Train-Guided Meta-Learning Framework for Robust and Scalable Variational Quantum Computing [60.996803677584424]
  TensoMeta-VQC is a novel tensor-train (TT)-guided meta-learning framework designed to improve the robustness and scalability of VQC significantly.<n>Our framework fully delegates the generation of quantum circuit parameters to a classical TT network, effectively decoupling optimization from quantum hardware.
  arXiv  Detail & Related papers  (2025-08-01T23:37:55Z)
• Quantum-Accelerated Neural Imputation with Large Language Models (LLMs) [0.0]
  This paper introduces Quantum-UnIMP, a novel framework that integrates shallow quantum circuits into an LLM-based imputation architecture.<n>Our experiments on benchmark mixed-type datasets demonstrate that Quantum-UnIMP reduces imputation error by up to 15.2% for numerical features (RMSE) and improves classification accuracy by 8.7% for categorical features (F1-Score) compared to state-of-the-art classical and LLM-based methods.
  arXiv  Detail & Related papers  (2025-07-11T02:00:06Z)
• Q2SAR: A Quantum Multiple Kernel Learning Approach for Drug Discovery [39.58317527488534]
  This research demonstrates the successful application of a Quantum Multiple Kernel Learning framework to enhance QSAR classification.<n>We apply this methodology to a dataset for identifying DYRK1A kinase inhibitors.<n>By benchmarking the QMKL-SVM against a classical Gradient Boosting model, we show that the quantum-enhanced approach achieves a superior AUC score.
  arXiv  Detail & Related papers  (2025-06-17T19:00:47Z)
• Evaluating Effects of Augmented SELFIES for Molecular Understanding Using QK-LSTM [2.348041867134616]
  Identifying molecular properties, including side effects, is a critical yet time-consuming step in drug development. Recent advancements have been made in the classical domain using augmented variations of the Simplified Molecular Line-Entry System (SMILES) This study presents the first analysis of these approaches, providing novel insights into their potential for enhancing molecular property prediction and side effect identification.
  arXiv  Detail & Related papers  (2025-04-29T14:03:31Z)
• Leveraging Pre-Trained Neural Networks to Enhance Machine Learning with Variational Quantum Circuits [48.33631905972908]
  We introduce an innovative approach that utilizes pre-trained neural networks to enhance Variational Quantum Circuits (VQC) This technique effectively separates approximation error from qubit count and removes the need for restrictive conditions. Our results extend to applications such as human genome analysis, demonstrating the broad applicability of our approach.
  arXiv  Detail & Related papers  (2024-11-13T12:03:39Z)
• Understanding Generalization in Quantum Machine Learning with Margins [0.46040036610482665]
  We present a margin-based generalization bound for QML models. By connecting this margin-based metric to quantum information theory, we demonstrate how to enhance the generalization performance of QML.
  arXiv  Detail & Related papers  (2024-11-11T12:22:18Z)
• Entanglement-enhanced optimal quantum metrology [0.7373617024876725]
  We propose a QOC scheme for QM that leverages entanglement and optimized coupling interactions with an ancillary system to provide enhanced metrological performance. Our findings indicate that, in certain situations, schemes employing coherent control of a single particle are severely limited.
  arXiv  Detail & Related papers  (2024-11-06T16:08:13Z)
• Benchmarking quantum machine learning kernel training for classification tasks [0.0]
  This study focuses on quantum kernel methods in the context of classification tasks. It examines the performance of Quantum Kernel Estimation (QKE) and Quantum Kernel Training (QKT) in connection with two quantum feature mappings. Experimental results indicate that quantum methods exhibit varying performance across different datasets.
  arXiv  Detail & Related papers  (2024-08-17T10:53:06Z)
• Quantum Markov Decision Processes: General Theory, Approximations, and Classes of Policies [1.8775413720750924]
  We present a novel quantum MDP model aiming to introduce a new framework, algorithms, and future research avenues. We hope that our approach will pave the way for a new research direction in discrete-time quantum control.
  arXiv  Detail & Related papers  (2024-02-22T15:59:09Z)
• Generating Universal Adversarial Perturbations for Quantum Classifiers [0.0]
  Quantum Machine Learning (QML) has emerged as a promising field of research, aiming to leverage the capabilities of quantum computing to enhance existing machine learning methodologies. Recent studies have revealed that, like their classical counterparts, QML models based on Parametrized Quantum Circuits (PQCs) are also vulnerable to adversarial attacks. We introduce QuGAP: a novel framework for generating Universal Adversarial Perturbations (UAPs) for quantum classifiers.
  arXiv  Detail & Related papers  (2024-02-13T18:27:53Z)
• Symmetric Pruning in Quantum Neural Networks [111.438286016951]
  Quantum neural networks (QNNs) exert the power of modern quantum machines. QNNs with handcraft symmetric ansatzes generally experience better trainability than those with asymmetric ansatzes. We propose the effective quantum neural tangent kernel (EQNTK) to quantify the convergence of QNNs towards the global optima.
  arXiv  Detail & Related papers  (2022-08-30T08:17:55Z)
• Twenty Years of Auxiliary-Field Quantum Monte Carlo in Quantum Chemistry: An Overview and Assessment on Main Group Chemistry and Bond-Breaking [0.6445605125467573]
  We present an overview of the phaseless auxiliary-field quantum Monte Carlo approach from a computational quantum chemistry perspective. We present a numerical assessment of its performance on main group chemistry and bond-breaking problems with a total of 1004 relative energies.
  arXiv  Detail & Related papers  (2022-08-02T07:02:44Z)
• Theory of Quantum Generative Learning Models with Maximum Mean Discrepancy [67.02951777522547]
  We study learnability of quantum circuit Born machines (QCBMs) and quantum generative adversarial networks (QGANs) We first analyze the generalization ability of QCBMs and identify their superiorities when the quantum devices can directly access the target distribution. Next, we prove how the generalization error bound of QGANs depends on the employed Ansatz, the number of qudits, and input states.
  arXiv  Detail & Related papers  (2022-05-10T08:05:59Z)
• Tight Mutual Information Estimation With Contrastive Fenchel-Legendre Optimization [69.07420650261649]
  We introduce a novel, simple, and powerful contrastive MI estimator named as FLO. Empirically, our FLO estimator overcomes the limitations of its predecessors and learns more efficiently. The utility of FLO is verified using an extensive set of benchmarks, which also reveals the trade-offs in practical MI estimation.
  arXiv  Detail & Related papers  (2021-07-02T15:20:41Z)
• Quantum circuit architecture search for variational quantum algorithms [88.71725630554758]
  We propose a resource and runtime efficient scheme termed quantum architecture search (QAS) QAS automatically seeks a near-optimal ansatz to balance benefits and side-effects brought by adding more noisy quantum gates. We implement QAS on both the numerical simulator and real quantum hardware, via the IBM cloud, to accomplish data classification and quantum chemistry tasks.
  arXiv  Detail & Related papers  (2020-10-20T12:06:27Z)
• Machine Learning Force Fields [54.48599172620472]
  Machine Learning (ML) has enabled numerous advances in computational chemistry. One of the most promising applications is the construction of ML-based force fields (FFs) This review gives an overview of applications of ML-FFs and the chemical insights that can be obtained from them.
  arXiv  Detail & Related papers  (2020-10-14T13:14:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.