The Quantum Cartpole: A benchmark environment for non-linear reinforcement learning

• URL: http://arxiv.org/abs/2311.00756v2
• Date: Sun, 10 Mar 2024 11:58:04 GMT
• Title: The Quantum Cartpole: A benchmark environment for non-linear reinforcement learning
• Authors: Kai Meinerz, Simon Trebst, Mark Rudner, Evert van Nieuwenburg
• Abstract summary: We show how a trade-off between state estimation and controllability arises. We demonstrate the feasibility of using transfer learning to develop a quantum control agent trained via reinforcement learning.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Feedback-based control is the de-facto standard when it comes to controlling classical stochastic systems and processes. However, standard feedback-based control methods are challenged by quantum systems due to measurement induced backaction and partial observability. Here we remedy this by using weak quantum measurements and model-free reinforcement learning agents to perform quantum control. By comparing control algorithms with and without state estimators to stabilize a quantum particle in an unstable state near a local potential energy maximum, we show how a trade-off between state estimation and controllability arises. For the scenario where the classical analogue is highly nonlinear, the reinforcement learned controller has an advantage over the standard controller. Additionally, we demonstrate the feasibility of using transfer learning to develop a quantum control agent trained via reinforcement learning on a classical surrogate of the quantum control problem. Finally, we present results showing how the reinforcement learning control strategy differs from the classical controller in the non-linear scenarios.

Related papers

• Quantum control by the environment: Turing uncomputability, Optimization over Stiefel manifolds, Reachable sets, and Incoherent GRAPE [56.47577824219207]
  In many practical situations, the controlled quantum systems are open, interacting with the environment. In this note, we briefly review some results on control of open quantum systems using environment as a resource.
  arXiv  Detail & Related papers  (2024-03-20T10:09:13Z)
• Effect of the readout efficiency of quantum measurement on the system entanglement [44.99833362998488]
  We quantify the entanglement for a particle on a 1d quantum random walk under inefficient monitoring. We find that the system's maximal mean entanglement at the measurement-induced quantum-to-classical crossover is in different ways by the measurement strength and inefficiency.
  arXiv  Detail & Related papers  (2024-02-29T18:10:05Z)
• A quantum system control method based on enhanced reinforcement learning [2.70857393901228]
  A quantum system control method based on enhanced reinforcement learning (QSC-ERL) is proposed. The states and actions in reinforcement learning are mapped to quantum states and control operations in quantum systems. Compared with other methods, the QSC-ERL achieves close to 1 fidelity learning control of quantum systems.
  arXiv  Detail & Related papers  (2023-09-30T03:22:44Z)
• Classical Verification of Quantum Learning [42.362388367152256]
  We develop a framework for classical verification of quantum learning. We propose a new quantum data access model that we call "mixture-of-superpositions" quantum examples. Our results demonstrate that the potential power of quantum data for learning tasks, while not unlimited, can be utilized by classical agents.
  arXiv  Detail & Related papers  (2023-06-08T00:31:27Z)
• Learning in quantum games [41.67943127631515]
  We show that the induced quantum state dynamics decompose into (i) a classical, commutative component which governs the dynamics of the system's eigenvalues. We find that the FTQL dynamics incur no more than constant regret in all quantum games.
  arXiv  Detail & Related papers  (2023-02-05T08:23:04Z)
• Robust optimization for quantum reinforcement learning control using partial observations [10.975734427172231]
  Full observation of quantum state is experimentally infeasible due to the exponential scaling of the number of required quantum measurements on the number of qubits. This control scheme is compatible with near-term quantum devices, where the noise is prevalent. It has been shown that high-fidelity state control can be achieved even if the noise amplitude is at the same level as the control amplitude.
  arXiv  Detail & Related papers  (2022-06-29T06:30:35Z)
• Self-Correcting Quantum Many-Body Control using Reinforcement Learning with Tensor Networks [0.0]
  We present a novel framework for efficiently controlling quantum many-body systems based on reinforcement learning (RL) We show that RL agents are capable of finding universal controls, of learning how to optimally steer previously unseen many-body states, and of adapting control protocols on-thefly when the quantum dynamics is subject to perturbations.
  arXiv  Detail & Related papers  (2022-01-27T20:14:09Z)
• Stochastic optimization for learning quantum state feedback control [16.4432244108711]
  We present a general framework for training deep feedback networks for open quantum systems with quantum nondemolition measurement. We demonstrate that this method is efficient due to inherent parallelizability, robust to open system interactions, and outperforms landmark state feedback control results in simulation.
  arXiv  Detail & Related papers  (2021-11-18T19:00:06Z)
• Sparsity in Partially Controllable Linear Systems [56.142264865866636]
  We study partially controllable linear dynamical systems specified by an underlying sparsity pattern. Our results characterize those state variables which are irrelevant for optimal control.
  arXiv  Detail & Related papers  (2021-10-12T16:41:47Z)
• Model-Free Quantum Control with Reinforcement Learning [0.0]
  We propose a circuit-based approach for training a reinforcement learning agent on quantum control tasks in a model-free way. We show how to reward the learning agent using measurements of experimentally available observables. This approach significantly outperforms widely used model-free methods in terms of sample efficiency.
  arXiv  Detail & Related papers  (2021-04-29T17:53:26Z)
• Closing the Closed-Loop Distribution Shift in Safe Imitation Learning [80.05727171757454]
  We treat safe optimization-based control strategies as experts in an imitation learning problem. We train a learned policy that can be cheaply evaluated at run-time and that provably satisfies the same safety guarantees as the expert.
  arXiv  Detail & Related papers  (2021-02-18T05:11:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.