Quantum Markov Decision Processes: General Theory, Approximations, and Classes of Policies

• URL: http://arxiv.org/abs/2402.14649v2
• Date: Wed, 18 Sep 2024 09:21:32 GMT
• Title: Quantum Markov Decision Processes: General Theory, Approximations, and Classes of Policies
• Authors: Naci Saldi, Sina Sanjari, Serdar Yuksel,
• Abstract summary: We present a novel quantum MDP model aiming to introduce a new framework, algorithms, and future research avenues. We hope that our approach will pave the way for a new research direction in discrete-time quantum control.
• Score: 1.8775413720750924
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, the aim is to develop a quantum counterpart to classical Markov decision processes (MDPs). Firstly, we provide a very general formulation of quantum MDPs with state and action spaces in the quantum domain, quantum transitions, and cost functions. Once we formulate the quantum MDP (q-MDP), our focus shifts to establishing the verification theorem that proves the sufficiency of Markovian quantum control policies and provides a dynamic programming principle. Subsequently, a comparison is drawn between our q-MDP model and previously established quantum MDP models (referred to as QOMDPs) found in the literature. Furthermore, approximations of q-MDPs are obtained via finite-action models, which can be formulated as QOMDPs. Finally, classes of open-loop and classical-state-preserving closed-loop policies for q-MDPs are introduced, along with structural results for these policies. In summary, we present a novel quantum MDP model aiming to introduce a new framework, algorithms, and future research avenues. We hope that our approach will pave the way for a new research direction in discrete-time quantum control.

Related papers

• Quantum Markov Decision Processes Part II: Optimal Solutions and Algorithms [1.8775413720750924]
  This two-part article aims to introduce a quantum analogue to classical Markov decision processes (MDPs) In Part II, our focus shifts to the development of algorithms for computing optimal policies and value functions of both open-loop and closed-loop policies.
  arXiv  Detail & Related papers  (2024-02-22T16:00:21Z)
• A new quantum machine learning algorithm: split hidden quantum Markov model inspired by quantum conditional master equation [14.262911696419934]
  We introduce the split HQMM (SHQMM) for implementing the hidden quantum Markov process. Experimental results suggest our model outperforms previous models in terms of scope of applications and robustness.
  arXiv  Detail & Related papers  (2023-07-17T16:55:26Z)
• A Framework for Demonstrating Practical Quantum Advantage: Racing Quantum against Classical Generative Models [62.997667081978825]
  We build over a proposed framework for evaluating the generalization performance of generative models. We establish the first comparative race towards practical quantum advantage (PQA) between classical and quantum generative models. Our results suggest that QCBMs are more efficient in the data-limited regime than the other state-of-the-art classical generative models.
  arXiv  Detail & Related papers  (2023-03-27T22:48:28Z)
• A self-consistent field approach for the variational quantum eigensolver: orbital optimization goes adaptive [52.77024349608834]
  We present a self consistent field approach (SCF) within the Adaptive Derivative-Assembled Problem-Assembled Ansatz Variational Eigensolver (ADAPTVQE) This framework is used for efficient quantum simulations of chemical systems on nearterm quantum computers.
  arXiv  Detail & Related papers  (2022-12-21T23:15:17Z)
• End-to-end resource analysis for quantum interior point methods and portfolio optimization [63.4863637315163]
  We provide a complete quantum circuit-level description of the algorithm from problem input to problem output. We report the number of logical qubits and the quantity/depth of non-Clifford T-gates needed to run the algorithm.
  arXiv  Detail & Related papers  (2022-11-22T18:54:48Z)
• Decomposition of Matrix Product States into Shallow Quantum Circuits [62.5210028594015]
  tensor network (TN) algorithms can be mapped to parametrized quantum circuits (PQCs) We propose a new protocol for approximating TN states using realistic quantum circuits. Our results reveal one particular protocol, involving sequential growth and optimization of the quantum circuit, to outperform all other methods.
  arXiv  Detail & Related papers  (2022-09-01T17:08:41Z)
• On the design and analysis of near-term quantum network protocols using Markov decision processes [0.9065034043031668]
  Theory of near-term quantum networks based on Markov decision processes (MDPs) MDPs provide a precise and systematic mathematical framework to model protocols for near-term quantum networks. New results show that the well-known memory-cutoff policy is optimal.
  arXiv  Detail & Related papers  (2022-07-07T16:18:59Z)
• Quantum Policy Iteration via Amplitude Estimation and Grover Search -- Towards Quantum Advantage for Reinforcement Learning [7.122914046030916]
  We show how to combine amplitude estimation and Grover search into a policy evaluation and improvement scheme. We derive a quantum policy iteration that repeatedly improves an initial policy using Grover search until the optimum is reached.
  arXiv  Detail & Related papers  (2022-06-09T19:23:10Z)
• Theory of Quantum Generative Learning Models with Maximum Mean Discrepancy [67.02951777522547]
  We study learnability of quantum circuit Born machines (QCBMs) and quantum generative adversarial networks (QGANs) We first analyze the generalization ability of QCBMs and identify their superiorities when the quantum devices can directly access the target distribution. Next, we prove how the generalization error bound of QGANs depends on the employed Ansatz, the number of qudits, and input states.
  arXiv  Detail & Related papers  (2022-05-10T08:05:59Z)
• Q-Learning for MDPs with General Spaces: Convergence and Near Optimality via Quantization under Weak Continuity [2.685668802278156]
  We show that Q-learning for standard Borel MDPs via quantization of states and actions converges to a limit. Our paper presents a very general convergence and approximation result for the applicability of Q-learning for continuous MDPs.
  arXiv  Detail & Related papers  (2021-11-12T15:47:10Z)
• State preparation and measurement in a quantum simulation of the O(3) sigma model [65.01359242860215]
  We show that fixed points of the non-linear O(3) sigma model can be reproduced near a quantum phase transition of a spin model with just two qubits per lattice site. We apply Trotter methods to obtain results for the complexity of adiabatic ground state preparation in both the weak-coupling and quantum-critical regimes. We present and analyze a quantum algorithm based on non-unitary randomized simulation methods.
  arXiv  Detail & Related papers  (2020-06-28T23:44:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.