Robust Quantum Control using Reinforcement Learning from Demonstration

• URL: http://arxiv.org/abs/2503.21085v2
• Date: Sun, 20 Apr 2025 03:39:39 GMT
• Title: Robust Quantum Control using Reinforcement Learning from Demonstration
• Authors: Shengyong Li, Yidian Fan, Xiang Li, Xinhui Ruan, Qianchuan Zhao, Zhihui Peng, Re-Bing Wu, Jing Zhang, Pengtao Song,
• Abstract summary: We use Reinforcement Learning from Demonstration (RLfD) to leverage the control sequences generated with system models.<n>This approach can increase sample efficiency by reducing the number of samples, which can significantly reduce the training time.<n>We have simulated the preparation of several high-fidelity non-classical states using the RLfD method.
• Score: 13.321147424579065
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Quantum control requires high-precision and robust control pulses to ensure optimal system performance. However, control sequences generated with a system model may suffer from model bias, leading to low fidelity. While model-free reinforcement learning (RL) methods have been developed to avoid such biases, training an RL agent from scratch can be time-consuming, often taking hours to gather enough samples for convergence. This challenge has hindered the broad application of RL techniques to larger and more complex quantum control issues, limiting their adaptability. In this work, we use Reinforcement Learning from Demonstration (RLfD) to leverage the control sequences generated with system models and further optimize them with RL to avoid model bias. By avoiding learning from scratch and starting with reasonable control pulse shapes, this approach can increase sample efficiency by reducing the number of samples, which can significantly reduce the training time. Thus, this method can effectively handle pulse shapes that are discretized into more than 1000 pieces without compromising final fidelity. We have simulated the preparation of several high-fidelity non-classical states using the RLfD method. We also find that the training process is more stable when using RLfD. In addition, this method is suitable for fast gate calibration using reinforcement learning.

Related papers

• Model-based controller assisted domain randomization in deep reinforcement learning: application to nonlinear powertrain control [0.0]
  This study proposes a new robust control approach using the framework of deep reinforcement learning (DRL) The problem setup is modeled via the latent Markov decision process (LMDP), a set of vanilla MDPs, for a controlled system subject to uncertainties and nonlinearities. Compared to traditional DRL-based controls, the proposed controller design is smarter in that we can achieve a high level of generalization ability.
  arXiv  Detail & Related papers  (2025-04-28T12:09:07Z)
• Controlling nonergodicity in quantum many-body systems by reinforcement learning [0.0]
  We develop a model-free and deep-reinforcement learning framework for quantum nonergodicity control.<n>We use the paradigmatic one-dimensional tilted Fermi-Hubbard system to demonstrate that the DRL agent can efficiently learn the quantum many-body system.<n>The continuous control protocols and observations are experimentally feasible.
  arXiv  Detail & Related papers  (2024-08-21T20:55:44Z)
• Adding Conditional Control to Diffusion Models with Reinforcement Learning [68.06591097066811]
  Diffusion models are powerful generative models that allow for precise control over the characteristics of the generated samples.<n>While these diffusion models trained on large datasets have achieved success, there is often a need to introduce additional controls in downstream fine-tuning processes.<n>This work presents a novel method based on reinforcement learning (RL) to add such controls using an offline dataset.
  arXiv  Detail & Related papers  (2024-06-17T22:00:26Z)
• Self-STORM: Deep Unrolled Self-Supervised Learning for Super-Resolution Microscopy [55.2480439325792]
  We introduce deep unrolled self-supervised learning, which alleviates the need for such data by training a sequence-specific, model-based autoencoder. Our proposed method exceeds the performance of its supervised counterparts.
  arXiv  Detail & Related papers  (2024-03-25T17:40:32Z)
• Value function estimation using conditional diffusion models for control [62.27184818047923]
  We propose a simple algorithm called Diffused Value Function (DVF) It learns a joint multi-step model of the environment-robot interaction dynamics using a diffusion model. We show how DVF can be used to efficiently capture the state visitation measure for multiple controllers.
  arXiv  Detail & Related papers  (2023-06-09T18:40:55Z)
• Direct Preference Optimization: Your Language Model is Secretly a Reward Model [119.65409513119963]
  We introduce a new parameterization of the reward model in RLHF that enables extraction of the corresponding optimal policy in closed form. The resulting algorithm, which we call Direct Preference Optimization (DPO), is stable, performant, and computationally lightweight. Our experiments show that DPO can fine-tune LMs to align with human preferences as well as or better than existing methods.
  arXiv  Detail & Related papers  (2023-05-29T17:57:46Z)
• Sample-efficient Model-based Reinforcement Learning for Quantum Control [0.2999888908665658]
  We propose a model-based reinforcement learning (RL) approach for noisy time-dependent gate optimization. We show an order of magnitude advantage in the sample complexity of our method over standard model-free RL. Our algorithm is well suited for controlling partially characterised one and two qubit systems.
  arXiv  Detail & Related papers  (2023-04-19T15:05:19Z)
• Real-Time Model-Free Deep Reinforcement Learning for Force Control of a Series Elastic Actuator [56.11574814802912]
  State-of-the art robotic applications utilize series elastic actuators (SEAs) with closed-loop force control to achieve complex tasks such as walking, lifting, and manipulation. Model-free PID control methods are more prone to instability due to nonlinearities in the SEA. Deep reinforcement learning has proved to be an effective model-free method for continuous control tasks.
  arXiv  Detail & Related papers  (2023-04-11T00:51:47Z)
• A stabilizing reinforcement learning approach for sampled systems with partially unknown models [0.0]
  We suggest a method to guarantee practical stability of the system-controller closed loop in a purely online learning setting. To achieve the claimed results, we employ techniques of classical adaptive control. The method is tested in adaptive traction control and cruise control where it proved to significantly reduce the cost.
  arXiv  Detail & Related papers  (2022-08-31T09:20:14Z)
• Supervised Advantage Actor-Critic for Recommender Systems [76.7066594130961]
  We propose negative sampling strategy for training the RL component and combine it with supervised sequential learning. Based on sampled (negative) actions (items), we can calculate the "advantage" of a positive action over the average case. We instantiate SNQN and SA2C with four state-of-the-art sequential recommendation models and conduct experiments on two real-world datasets.
  arXiv  Detail & Related papers  (2021-11-05T12:51:15Z)
• Model-Free Quantum Control with Reinforcement Learning [0.0]
  We propose a circuit-based approach for training a reinforcement learning agent on quantum control tasks in a model-free way. We show how to reward the learning agent using measurements of experimentally available observables. This approach significantly outperforms widely used model-free methods in terms of sample efficiency.
  arXiv  Detail & Related papers  (2021-04-29T17:53:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.