Model-Free Quantum Control with Reinforcement Learning

• URL: http://arxiv.org/abs/2104.14539v2
• Date: Mon, 6 Dec 2021 18:45:04 GMT
• Title: Model-Free Quantum Control with Reinforcement Learning
• Authors: V. V. Sivak, A. Eickbusch, H. Liu, B. Royer, I. Tsioutsios, M. H. Devoret
• Abstract summary: We propose a circuit-based approach for training a reinforcement learning agent on quantum control tasks in a model-free way. We show how to reward the learning agent using measurements of experimentally available observables. This approach significantly outperforms widely used model-free methods in terms of sample efficiency.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Model bias is an inherent limitation of the current dominant approach to optimal quantum control, which relies on a system simulation for optimization of control policies. To overcome this limitation, we propose a circuit-based approach for training a reinforcement learning agent on quantum control tasks in a model-free way. Given a continuously parameterized control circuit, the agent learns its parameters through trial-and-error interaction with the quantum system, using measurement outcomes as the only source of information about the quantum state. Focusing on control of a harmonic oscillator coupled to an ancilla qubit, we show how to reward the learning agent using measurements of experimentally available observables. We train the agent to prepare various non-classical states using both unitary control and control with adaptive measurement-based quantum feedback, and to execute logical gates on encoded qubits. This approach significantly outperforms widely used model-free methods in terms of sample efficiency. Our numerical work is of immediate relevance to superconducting circuits and trapped ions platforms where such training can be implemented in experiment, allowing complete elimination of model bias and the adaptation of quantum control policies to the specific system in which they are deployed.

Related papers

• Controlling nonergodicity in quantum many-body systems by reinforcement learning [0.0]
  We develop a model-free and deep-reinforcement learning framework for quantum nonergodicity control. We use the paradigmatic one-dimensional tilted Fermi-Hubbard system to demonstrate that the DRL agent can efficiently learn the quantum many-body system. The continuous control protocols and observations are experimentally feasible.
  arXiv  Detail & Related papers  (2024-08-21T20:55:44Z)
• Optimal control in large open quantum systems: the case of transmon readout and reset [44.99833362998488]
  We present a framework that combines the adjoint state method together with reverse-time back-propagation to solve prohibitively large open-system quantum control problems. We apply this framework to optimize two inherently dissipative operations in superconducting qubits. Our results show that, given a fixed set of system parameters, shaping the control pulses can yield 2x improvements in the fidelity and duration for both of these operations.
  arXiv  Detail & Related papers  (2024-03-21T18:12:51Z)
• Quantum control by the environment: Turing uncomputability, Optimization over Stiefel manifolds, Reachable sets, and Incoherent GRAPE [56.47577824219207]
  In many practical situations, the controlled quantum systems are open, interacting with the environment. In this note, we briefly review some results on control of open quantum systems using environment as a resource.
  arXiv  Detail & Related papers  (2024-03-20T10:09:13Z)
• Non-Markovian Quantum Control via Model Maximum Likelihood Estimation and Reinforcement Learning [0.0]
  We propose a novel approach that incorporates the non-Markovian nature of the environment into a low-dimensional effective reservoir. We utilize machine learning techniques to learn the effective quantum dynamics more efficiently than traditional tomographic methods. This approach may not only mitigates the issues of model bias but also provides a more accurate representation of quantum dynamics.
  arXiv  Detail & Related papers  (2024-02-07T18:37:17Z)
• The Quantum Cartpole: A benchmark environment for non-linear reinforcement learning [0.0]
  We show how a trade-off between state estimation and controllability arises. We demonstrate the feasibility of using transfer learning to develop a quantum control agent trained via reinforcement learning.
  arXiv  Detail & Related papers  (2023-11-01T18:02:42Z)
• Robust optimization for quantum reinforcement learning control using partial observations [10.975734427172231]
  Full observation of quantum state is experimentally infeasible due to the exponential scaling of the number of required quantum measurements on the number of qubits. This control scheme is compatible with near-term quantum devices, where the noise is prevalent. It has been shown that high-fidelity state control can be achieved even if the noise amplitude is at the same level as the control amplitude.
  arXiv  Detail & Related papers  (2022-06-29T06:30:35Z)
• Generalization Metrics for Practical Quantum Advantage in Generative Models [68.8204255655161]
  Generative modeling is a widely accepted natural use case for quantum computers. We construct a simple and unambiguous approach to probe practical quantum advantage for generative modeling by measuring the algorithm's generalization performance. Our simulation results show that our quantum-inspired models have up to a $68 times$ enhancement in generating unseen unique and valid samples.
  arXiv  Detail & Related papers  (2022-01-21T16:35:35Z)
• Dynamical learning of a photonics quantum-state engineering process [48.7576911714538]
  Experimentally engineering high-dimensional quantum states is a crucial task for several quantum information protocols. We implement an automated adaptive optimization protocol to engineer photonic Orbital Angular Momentum (OAM) states. This approach represents a powerful tool for automated optimizations of noisy experimental tasks for quantum information protocols and technologies.
  arXiv  Detail & Related papers  (2022-01-14T19:24:31Z)
• Model predictive control for robust quantum state preparation [4.069849286089743]
  We introduce model predictive control (MPC) for quantum control applications. MPC inherits a natural degree of disturbance rejection by incorporating measurement feedback. We show how MPC can be used to generate practical optimized control sequences.
  arXiv  Detail & Related papers  (2022-01-14T00:55:41Z)
• Sparsity in Partially Controllable Linear Systems [56.142264865866636]
  We study partially controllable linear dynamical systems specified by an underlying sparsity pattern. Our results characterize those state variables which are irrelevant for optimal control.
  arXiv  Detail & Related papers  (2021-10-12T16:41:47Z)
• Error mitigation and quantum-assisted simulation in the error corrected regime [77.34726150561087]
  A standard approach to quantum computing is based on the idea of promoting a classically simulable and fault-tolerant set of operations. We show how the addition of noisy magic resources allows one to boost classical quasiprobability simulations of a quantum circuit.
  arXiv  Detail & Related papers  (2021-03-12T20:58:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.