Model-aware reinforcement learning for high-performance Bayesian experimental design in quantum metrology

• URL: http://arxiv.org/abs/2312.16985v3
• Date: Tue, 03 Dec 2024 16:13:04 GMT
• Title: Model-aware reinforcement learning for high-performance Bayesian experimental design in quantum metrology
• Authors: Federico Belliardo, Fabio Zoratti, Florian Marquardt, Vittorio Giovannetti,
• Abstract summary: Quantum sensors offer control flexibility during estimation by allowing manipulation by the experimenter across various parameters. We introduce a versatile procedure capable of optimizing a wide range of problems in quantum metrology, estimation, and hypothesis testing. We combine model-aware reinforcement learning (RL) with Bayesian estimation based on particle filtering.
• Score: 0.4999814847776097
• License:
• Abstract: Quantum sensors offer control flexibility during estimation by allowing manipulation by the experimenter across various parameters. For each sensing platform, pinpointing the optimal controls to enhance the sensor's precision remains a challenging task. While an analytical solution might be out of reach, machine learning offers a promising avenue for many systems of interest, especially given the capabilities of contemporary hardware. We have introduced a versatile procedure capable of optimizing a wide range of problems in quantum metrology, estimation, and hypothesis testing by combining model-aware reinforcement learning (RL) with Bayesian estimation based on particle filtering. To achieve this, we had to address the challenge of incorporating the many non-differentiable steps of the estimation in the training process, such as measurements and the resampling of the particle filter. Model-aware RL is a gradient-based method, where the derivatives of the sensor's precision are obtained through automatic differentiation (AD) in the simulation of the experiment. Our approach is suitable for optimizing both non-adaptive and adaptive strategies, using neural networks or other agents. We provide an implementation of this technique in the form of a Python library called qsensoropt, alongside several pre-made applications for relevant physical platforms, namely NV centers, photonic circuits, and optical cavities. This library will be released soon on PyPI. Leveraging our method, we've achieved results for many examples that surpass the current state-of-the-art in experimental design. In addition to Bayesian estimation, leveraging model-aware RL, it is also possible to find optimal controls for the minimization of the Cram\'er-Rao bound, based on Fisher information.

Related papers

• Synergistic Development of Perovskite Memristors and Algorithms for Robust Analog Computing [53.77822620185878]
  We propose a synergistic methodology to concurrently optimize perovskite memristor fabrication and develop robust analog DNNs. We develop "BayesMulti", a training strategy utilizing BO-guided noise injection to improve the resistance of analog DNNs to memristor imperfections. Our integrated approach enables use of analog computing in much deeper and wider networks, achieving up to 100-fold improvements.
  arXiv  Detail & Related papers  (2024-12-03T19:20:08Z)
• Applications of model-aware reinforcement learning in Bayesian quantum metrology [0.6138671548064355]
  In Belliardo et al., arXiv:2312.16985 (2023), we introduced a procedure capable of optimizing a wide range of tasks in quantum metrology and estimation. We take a model-based approach to the optimisation where the physics describing the system is explicitly taken into account in the training. This exposition showcases the broad applicability of the method, which has been implemented in the qsensoropt library released on PyPI.
  arXiv  Detail & Related papers  (2024-03-08T22:48:07Z)
• Deep Bayesian Experimental Design for Quantum Many-Body Systems [0.0]
  We show how this approach holds promise for adaptive measurement strategies to characterize present-day quantum technology platforms. In particular, we focus on arrays of coupled cavities and qubit arrays.
  arXiv  Detail & Related papers  (2023-06-26T08:40:14Z)
• Machine learning enabled experimental design and parameter estimation for ultrafast spin dynamics [54.172707311728885]
  We introduce a methodology that combines machine learning with Bayesian optimal experimental design (BOED) Our method employs a neural network model for large-scale spin dynamics simulations for precise distribution and utility calculations in BOED. Our numerical benchmarks demonstrate the superior performance of our method in guiding XPFS experiments, predicting model parameters, and yielding more informative measurements within limited experimental time.
  arXiv  Detail & Related papers  (2023-06-03T06:19:20Z)
• CAS4DL: Christoffel Adaptive Sampling for function approximation via Deep Learning [2.513785998932353]
  We propose an adaptive sampling strategy, CAS4DL, to increase the sample efficiency of Deep Learning (DL) Our results show that CAS4DL often yields substantial savings in the number of samples required to achieve a given accuracy.
  arXiv  Detail & Related papers  (2022-08-25T16:21:17Z)
• Few-shot Quality-Diversity Optimization [50.337225556491774]
  Quality-Diversity (QD) optimization has been shown to be effective tools in dealing with deceptive minima and sparse rewards in Reinforcement Learning. We show that, given examples from a task distribution, information about the paths taken by optimization in parameter space can be leveraged to build a prior population, which when used to initialize QD methods in unseen environments, allows for few-shot adaptation. Experiments carried in both sparse and dense reward settings using robotic manipulation and navigation benchmarks show that it considerably reduces the number of generations that are required for QD optimization in these environments.
  arXiv  Detail & Related papers  (2021-09-14T17:12:20Z)
• Gone Fishing: Neural Active Learning with Fisher Embeddings [55.08537975896764]
  There is an increasing need for active learning algorithms that are compatible with deep neural networks. This article introduces BAIT, a practical representation of tractable, and high-performing active learning algorithm for neural networks.
  arXiv  Detail & Related papers  (2021-06-17T17:26:31Z)
• PlasticineLab: A Soft-Body Manipulation Benchmark with Differentiable Physics [89.81550748680245]
  We introduce a new differentiable physics benchmark called PasticineLab. In each task, the agent uses manipulators to deform the plasticine into the desired configuration. We evaluate several existing reinforcement learning (RL) methods and gradient-based methods on this benchmark.
  arXiv  Detail & Related papers  (2021-04-07T17:59:23Z)
• Large-scale Neural Solvers for Partial Differential Equations [48.7576911714538]
  Solving partial differential equations (PDE) is an indispensable part of many branches of science as many processes can be modelled in terms of PDEs. Recent numerical solvers require manual discretization of the underlying equation as well as sophisticated, tailored code for distributed computing. We examine the applicability of continuous, mesh-free neural solvers for partial differential equations, physics-informed neural networks (PINNs) We discuss the accuracy of GatedPINN with respect to analytical solutions -- as well as state-of-the-art numerical solvers, such as spectral solvers.
  arXiv  Detail & Related papers  (2020-09-08T13:26:51Z)
• Single-step deep reinforcement learning for open-loop control of laminar and turbulent flows [0.0]
  This research gauges the ability of deep reinforcement learning (DRL) techniques to assist the optimization and control of fluid mechanical systems. It combines a novel, "degenerate" version of the prototypical policy optimization (PPO) algorithm, that trains a neural network in optimizing the system only once per learning episode.
  arXiv  Detail & Related papers  (2020-06-04T16:11:26Z)
• A machine learning approach to Bayesian parameter estimation [0.0]
  We formulate parameter estimation as a classification task and use artificial neural networks to efficiently perform Bayesian estimation. We show that the network's posterior distribution is centered at the true (unknown) value of the parameter within an uncertainty given by the inverse Fisher information.
  arXiv  Detail & Related papers  (2020-06-03T16:33:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.