Model-based Offline Quantum Reinforcement Learning

• URL: http://arxiv.org/abs/2404.10017v1
• Date: Sun, 14 Apr 2024 15:11:27 GMT
• Title: Model-based Offline Quantum Reinforcement Learning
• Abstract summary: This paper presents the first algorithm for model-based offline quantum reinforcement learning. It gives hope that a quantum advantage can be achieved as soon as sufficiently powerful quantum computers are available.
• Score: 4.912318087940015
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper presents the first algorithm for model-based offline quantum reinforcement learning and demonstrates its functionality on the cart-pole benchmark. The model and the policy to be optimized are each implemented as variational quantum circuits. The model is trained by gradient descent to fit a pre-recorded data set. The policy is optimized with a gradient-free optimization scheme using the return estimate given by the model as the fitness function. This model-based approach allows, in principle, full realization on a quantum computer during the optimization phase and gives hope that a quantum advantage can be achieved as soon as sufficiently powerful quantum computers are available.

Related papers

• A Model Predictive Control-Inspired Quantum Algorithm [0.0]
  We introduce a new hybrid quantum-classical algorithm inspired by an advanced control strategy known as model predictive control (MPC)<n>This algorithm unifies the optimization-based design of variational quantum algorithms (VQAs) with the feedback-based design of feedback-based quantum algorithms (FQAs)<n>We show through mathematical proof and numerical evidence that the MPC-based algorithm can be guaranteed to at least match the performance of FQAs.
  arXiv  Detail & Related papers  (2026-07-27T18:45:13Z)
• Stochastic Pauli-path simulator for large-scale quantum optimization [57.179828941706006]
  Pauli-path simulators offer a promising route to large-scale classical simulation of quantum circuits in the low-magic regime.<n>Existing approaches either lack native support for gradient-based optimization or suffer from severe gradient bias.<n>Here we propose the Pauli-path simulator (SPPS), a computational framework for large-scale quantum optimization.
  arXiv  Detail & Related papers  (2026-07-20T10:43:44Z)
• Symbolic Pauli Propagation for Gradient-Enabled Pre-Training of Quantum Circuits [0.0]
  Quantum Machine Learning models typically require expensive on-chip training procedures and often lack efficient gradient estimation methods.<n>By employing Pauli propagation, it is possible to derive a symbolic representation of observables as analytic functions of a circuit's parameters.<n>The proposed approach is demonstrated on the Variational Quantum Eigensolver for obtaining the ground state of a spin model.
  arXiv  Detail & Related papers  (2025-12-18T15:44:07Z)
• Sample-based training of quantum generative models [1.3521721488318912]
  We introduce a training framework that extends the principle of contrastive divergence to quantum models.<n>By deriving the circuit structure and providing a general recipe for constructing it, we obtain quantum circuits that generate the samples required for parameter updates.
  arXiv  Detail & Related papers  (2025-11-14T19:00:02Z)
• Reinforcement Learning for Quantum Network Control with Application-Driven Objectives [53.03367590211247]
  Dynamic programming and reinforcement learning offer promising tools for optimizing control strategies.<n>We propose a novel RL framework that directly optimize non-linear, differentiable objective functions.<n>Our work comprises the first step towards non-linear objective function optimization in quantum networks with RL, opening a path towards more advanced use cases.
  arXiv  Detail & Related papers  (2025-09-12T18:41:10Z)
• Q-MAML: Quantum Model-Agnostic Meta-Learning for Variational Quantum Algorithms [4.525216077859531]
  We introduce a new framework for optimizing parameterized quantum circuits (PQCs) that employs a classical, inspired by Model-Agnostic Meta-Learning (MAML) technique. Our framework features a classical neural network, called Learner, which interacts with a PQC using the output of Learner as an initial parameter. In the adaptation phase, the framework requires only a few PQC updates to converge to a more accurate value, while the learner remains unchanged.
  arXiv  Detail & Related papers  (2025-01-10T12:07:00Z)
• Adiabatic training for Variational Quantum Algorithms [0.4374837991804085]
  This paper presents a new hybrid Quantum Machine Learning (QML) model composed of three elements. Gate-based Quantum Computer running the Variational Quantum Algorithm (VQA) representing the Quantum Neural Network (QNN) An adiabatic Quantum Computer where the optimization function is executed to find the best parameters for the VQA.
  arXiv  Detail & Related papers  (2024-10-24T10:17:48Z)
• Learning Parameterized Quantum Circuits with Quantum Gradient [8.64967968665265]
  We introduce a nested optimization model that leverages quantum gradient to enhance PQC learning for gradient-type cost functions. Our approach utilizes quantum algorithms to identify and overcome a type of gradient vanishing-a persistent challenge in PQC learning.
  arXiv  Detail & Related papers  (2024-09-30T07:50:47Z)
• Dissipative learning of a quantum classifier [0.0]
  We analyze the learning dynamics of a quantum classifier model that works as an open quantum system. The model can be successfully trained with a gradient descent (GD) based algorithm. The fact that these optimization processes have been obtained with continuous dynamics, shows promise for the development of a differentiable activation function.
  arXiv  Detail & Related papers  (2023-07-23T11:08:51Z)
• A self-consistent field approach for the variational quantum eigensolver: orbital optimization goes adaptive [52.77024349608834]
  We present a self consistent field approach (SCF) within the Adaptive Derivative-Assembled Problem-Assembled Ansatz Variational Eigensolver (ADAPTVQE) This framework is used for efficient quantum simulations of chemical systems on nearterm quantum computers.
  arXiv  Detail & Related papers  (2022-12-21T23:15:17Z)
• QuACK: Accelerating Gradient-Based Quantum Optimization with Koopman Operator Learning [4.134992977596645]
  We present Quantum-circuit Controlled Alternating Koopman learning (QuACK), a novel framework that leverages an alternating algorithm for efficient prediction of gradient dynamics on quantum computers. We demonstrate QuACK's remarkable ability to accelerate gradient-based optimization across a range of applications in quantum optimization and machine learning.
  arXiv  Detail & Related papers  (2022-11-02T17:59:25Z)
• Evaluating the Convergence of Tabu Enhanced Hybrid Quantum Optimization [58.720142291102135]
  We introduce the Tabu Enhanced Hybrid Quantum Optimization metaheuristic approach useful for optimization problem solving on a quantum hardware. We address the theoretical convergence of the proposed scheme from the viewpoint of the collisions in the object which stores the tabu states, based on the Ising model.
  arXiv  Detail & Related papers  (2022-09-05T07:23:03Z)
• Generalization Metrics for Practical Quantum Advantage in Generative Models [68.8204255655161]
  Generative modeling is a widely accepted natural use case for quantum computers. We construct a simple and unambiguous approach to probe practical quantum advantage for generative modeling by measuring the algorithm's generalization performance. Our simulation results show that our quantum-inspired models have up to a $68 times$ enhancement in generating unseen unique and valid samples.
  arXiv  Detail & Related papers  (2022-01-21T16:35:35Z)
• Quantum algorithms for quantum dynamics: A performance study on the spin-boson model [68.8204255655161]
  Quantum algorithms for quantum dynamics simulations are traditionally based on implementing a Trotter-approximation of the time-evolution operator. variational quantum algorithms have become an indispensable alternative, enabling small-scale simulations on present-day hardware. We show that, despite providing a clear reduction of quantum gate cost, the variational method in its current implementation is unlikely to lead to a quantum advantage.
  arXiv  Detail & Related papers  (2021-08-09T18:00:05Z)
• Optimized Low-Depth Quantum Circuits for Molecular Electronic Structure using a Separable Pair Approximation [0.0]
  We present a classically solvable model that leads to optimized low-depth quantum circuits leveraging separable pair approximations. The obtained circuits are well suited as a baseline circuit for emerging quantum hardware and can, in the long term, provide significantly improved initial states for quantum algorithms.
  arXiv  Detail & Related papers  (2021-05-09T05:10:59Z)
• Quantum Machine Learning with SQUID [64.53556573827525]
  We present the Scaled QUantum IDentifier (SQUID), an open-source framework for exploring hybrid Quantum-Classical algorithms for classification problems. We provide examples of using SQUID in a standard binary classification problem from the popular MNIST dataset.
  arXiv  Detail & Related papers  (2021-04-30T21:34:11Z)
• Benchmarking adaptive variational quantum eigensolvers [63.277656713454284]
  We benchmark the accuracy of VQE and ADAPT-VQE to calculate the electronic ground states and potential energy curves. We find both methods provide good estimates of the energy and ground state. gradient-based optimization is more economical and delivers superior performance than analogous simulations carried out with gradient-frees.
  arXiv  Detail & Related papers  (2020-11-02T19:52:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.